Command line
python /home/saxelrod/Repo/projects/chemprop/chemprop/train.py --config_path /home/saxelrod/chemprop_cov_2/models/from_cp_feats_mpnn_from_binary_cross_entropy/config.json --data_path /home/saxelrod/chemprop_cov_2/scaffold_split/train_full.csv --dataset_type classification
Args
{'activation': 'ReLU',
 'aggregation': 'mean',
 'aggregation_norm': 100,
 'atom_descriptors': None,
 'atom_descriptors_path': None,
 'atom_descriptors_size': 0,
 'atom_features_size': 0,
 'atom_messages': False,
 'batch_size': 50,
 'bias': False,
 'cache_cutoff': 10000,
 'checkpoint_dir': None,
 'checkpoint_path': None,
 'checkpoint_paths': None,
 'class_balance': True,
 'config_path': '/home/saxelrod/chemprop_cov_2/models/from_cp_feats_mpnn_from_binary_cross_entropy/config.json',
 'crossval_index_dir': None,
 'crossval_index_file': None,
 'crossval_index_sets': None,
 'cuda': True,
 'data_path': '/home/saxelrod/chemprop_cov_2/scaffold_split/train_full.csv',
 'dataset_type': 'classification',
 'depth': 6,
 'device': device(type='cuda', index=1),
 'dropout': 0.4,
 'ensemble_size': 1,
 'epochs': 300,
 'extra_metrics': [],
 'features_generator': None,
 'features_only': False,
 'features_path': ['/home/saxelrod/chemprop_cov_2/features/from_cp/train_binary_cross_entropy.npz'],
 'features_scaling': False,
 'features_size': None,
 'ffn_hidden_size': 800,
 'ffn_num_layers': 3,
 'final_lr': 0.0001,
 'folds_file': None,
 'gpu': 1,
 'grad_clip': None,
 'hidden_size': 800,
 'ignore_columns': None,
 'init_lr': 0.0001,
 'log_frequency': 10,
 'max_data_size': None,
 'max_lr': 0.001,
 'metric': 'binary_cross_entropy',
 'metrics': ['binary_cross_entropy'],
 'minimize_score': True,
 'mpn_shared': False,
 'multiclass_num_classes': 3,
 'no_cache_mol': False,
 'no_cuda': False,
 'no_features_scaling': True,
 'num_folds': 10,
 'num_lrs': 1,
 'num_tasks': 1,
 'num_workers': 8,
 'number_of_molecules': 1,
 'pytorch_seed': 0,
 'quiet': True,
 'save_dir': '/home/saxelrod/chemprop_cov_2/models/from_cp_feats_mpnn_from_binary_cross_entropy',
 'save_preds': False,
 'save_smiles_splits': False,
 'seed': 0,
 'separate_test_features_path': ['/home/saxelrod/chemprop_cov_2/features/from_cp/test_binary_cross_entropy.npz'],
 'separate_test_path': '/home/saxelrod/chemprop_cov_2/scaffold_split/test_full.csv',
 'separate_val_features_path': ['/home/saxelrod/chemprop_cov_2/features/from_cp/val_binary_cross_entropy.npz'],
 'separate_val_path': '/home/saxelrod/chemprop_cov_2/scaffold_split/val_full.csv',
 'show_individual_scores': False,
 'smiles_columns': [None],
 'split_sizes': (0.8, 0.1, 0.1),
 'split_type': 'random',
 'target_columns': None,
 'task_names': ['sars_cov_two_cl_protease_active'],
 'test': False,
 'test_fold_index': None,
 'train_data_size': None,
 'undirected': False,
 'use_input_features': True,
 'val_fold_index': None,
 'warmup_epochs': 2.0}
Loading data
Number of tasks = 1
Fold 0
Splitting data with seed 0
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.4, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=800, bias=False)
        (W_h): Linear(in_features=800, out_features=800, bias=False)
        (W_o): Linear(in_features=933, out_features=800, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.4, inplace=False)
    (1): Linear(in_features=3100, out_features=800, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.4, inplace=False)
    (4): Linear(in_features=800, out_features=800, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.4, inplace=False)
    (7): Linear(in_features=800, out_features=1, bias=True)
  )
)
Number of parameters = 4,627,201
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 0.291513
Epoch 1
Validation binary_cross_entropy = 0.300563
Epoch 2
Validation binary_cross_entropy = 0.549332
Epoch 3
Validation binary_cross_entropy = 0.331805
Epoch 4
Loss = 8.2302e-01, PNorm = 63.1518, GNorm = 7.3082, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 0.310001
Epoch 5
Validation binary_cross_entropy = 0.413272
Epoch 6
Validation binary_cross_entropy = 0.632496
Epoch 7
Validation binary_cross_entropy = 0.291358
Epoch 8
Validation binary_cross_entropy = 0.298074
Epoch 9
Loss = 7.6393e-01, PNorm = 63.2881, GNorm = 5.3321, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 0.638309
Epoch 10
Validation binary_cross_entropy = 0.450271
Epoch 11
Validation binary_cross_entropy = 0.313449
Epoch 12
Validation binary_cross_entropy = 0.491421
Epoch 13
Validation binary_cross_entropy = 0.476504
Epoch 14
Loss = 8.1098e-01, PNorm = 63.4439, GNorm = 4.9537, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 0.327053
Epoch 15
Validation binary_cross_entropy = 0.434943
Epoch 16
Validation binary_cross_entropy = 0.539258
Epoch 17
Validation binary_cross_entropy = 0.415537
Epoch 18
Validation binary_cross_entropy = 0.527824
Epoch 19
Loss = 6.9978e-01, PNorm = 63.5571, GNorm = 4.2303, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 0.408644
Epoch 20
Validation binary_cross_entropy = 0.369876
Epoch 21
Validation binary_cross_entropy = 0.589452
Epoch 22
Validation binary_cross_entropy = 0.481779
Epoch 23
Validation binary_cross_entropy = 0.485017
Epoch 24
Loss = 6.6369e-01, PNorm = 63.6398, GNorm = 2.8893, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 0.437743
Epoch 25
Validation binary_cross_entropy = 0.585645
Epoch 26
Validation binary_cross_entropy = 0.537080
Epoch 27
Validation binary_cross_entropy = 0.552320
Epoch 28
Validation binary_cross_entropy = 0.499481
Epoch 29
Loss = 5.9060e-01, PNorm = 63.7167, GNorm = 2.8534, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 0.480392
Epoch 30
Validation binary_cross_entropy = 0.443153
Epoch 31
Validation binary_cross_entropy = 0.530520
Epoch 32
Validation binary_cross_entropy = 0.685076
Epoch 33
Validation binary_cross_entropy = 0.514649
Epoch 34
Loss = 7.3160e-01, PNorm = 63.8002, GNorm = 5.0103, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 0.427233
Epoch 35
Validation binary_cross_entropy = 0.507239
Epoch 36
Validation binary_cross_entropy = 0.536830
Epoch 37
Validation binary_cross_entropy = 0.542240
Epoch 38
Validation binary_cross_entropy = 0.477961
Epoch 39
Loss = 5.9979e-01, PNorm = 63.8617, GNorm = 2.2587, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 0.490366
Epoch 40
Validation binary_cross_entropy = 0.501424
Epoch 41
Validation binary_cross_entropy = 0.493479
Epoch 42
Validation binary_cross_entropy = 0.478281
Epoch 43
Validation binary_cross_entropy = 0.497873
Epoch 44
Loss = 6.4639e-01, PNorm = 63.9125, GNorm = 2.9476, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 0.497061
Epoch 45
Validation binary_cross_entropy = 0.504444
Epoch 46
Validation binary_cross_entropy = 0.524794
Epoch 47
Validation binary_cross_entropy = 0.565214
Epoch 48
Validation binary_cross_entropy = 0.571733
Epoch 49
Loss = 5.7608e-01, PNorm = 63.9690, GNorm = 2.0715, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 0.566454
Epoch 50
Validation binary_cross_entropy = 0.489663
Epoch 51
Validation binary_cross_entropy = 0.466363
Epoch 52
Validation binary_cross_entropy = 0.527071
Epoch 53
Validation binary_cross_entropy = 0.585976
Epoch 54
Loss = 5.8842e-01, PNorm = 64.0205, GNorm = 1.6390, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 0.606649
Epoch 55
Validation binary_cross_entropy = 0.569462
Epoch 56
Validation binary_cross_entropy = 0.489244
Epoch 57
Validation binary_cross_entropy = 0.450669
Epoch 58
Validation binary_cross_entropy = 0.498745
Epoch 59
Loss = 5.4210e-01, PNorm = 64.0860, GNorm = 2.1165, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 0.594303
Epoch 60
Validation binary_cross_entropy = 0.565601
Epoch 61
Validation binary_cross_entropy = 0.526246
Epoch 62
Validation binary_cross_entropy = 0.553257
Epoch 63
Validation binary_cross_entropy = 0.365143
Epoch 64
Loss = 5.6708e-01, PNorm = 64.1581, GNorm = 3.8693, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 0.424702
Epoch 65
Validation binary_cross_entropy = 0.734818
Epoch 66
Validation binary_cross_entropy = 0.593481
Epoch 67
Validation binary_cross_entropy = 0.479614
Epoch 68
Validation binary_cross_entropy = 0.436375
Epoch 69
Loss = 4.8166e-01, PNorm = 64.2429, GNorm = 1.3508, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 0.496648
Epoch 70
Validation binary_cross_entropy = 0.448676
Epoch 71
Validation binary_cross_entropy = 0.368955
Epoch 72
Validation binary_cross_entropy = 0.416038
Epoch 73
Validation binary_cross_entropy = 0.405116
Epoch 74
Loss = 4.6212e-01, PNorm = 64.3381, GNorm = 1.7301, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 0.385286
Epoch 75
Validation binary_cross_entropy = 0.317430
Epoch 76
Validation binary_cross_entropy = 0.640763
Epoch 77
Validation binary_cross_entropy = 0.456983
Epoch 78
Validation binary_cross_entropy = 0.391593
Epoch 79
Loss = 5.0234e-01, PNorm = 64.4190, GNorm = 2.3067, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 0.424095
Epoch 80
Validation binary_cross_entropy = 0.564847
Epoch 81
Validation binary_cross_entropy = 0.729193
Epoch 82
Validation binary_cross_entropy = 0.543196
Epoch 83
Validation binary_cross_entropy = 0.400939
Epoch 84
Loss = 5.0515e-01, PNorm = 64.5331, GNorm = 2.1307, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 0.419389
Epoch 85
Validation binary_cross_entropy = 0.588215
Epoch 86
Validation binary_cross_entropy = 0.423592
Epoch 87
Validation binary_cross_entropy = 0.382869
Epoch 88
Validation binary_cross_entropy = 0.428927
Epoch 89
Loss = 4.7803e-01, PNorm = 64.6275, GNorm = 1.5439, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 0.550175
Epoch 90
Validation binary_cross_entropy = 0.486576
Epoch 91
Validation binary_cross_entropy = 0.395095
Epoch 92
Validation binary_cross_entropy = 0.368515
Epoch 93
Validation binary_cross_entropy = 0.364157
Epoch 94
Loss = 3.8211e-01, PNorm = 64.7266, GNorm = 1.6379, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 0.492058
Epoch 95
Validation binary_cross_entropy = 0.335302
Epoch 96
Validation binary_cross_entropy = 0.334527
Epoch 97
Validation binary_cross_entropy = 0.423339
Epoch 98
Validation binary_cross_entropy = 0.435534
Epoch 99
Loss = 4.0832e-01, PNorm = 64.8207, GNorm = 1.9100, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.332675
Epoch 100
Validation binary_cross_entropy = 0.340582
Epoch 101
Validation binary_cross_entropy = 0.458380
Epoch 102
Validation binary_cross_entropy = 0.325717
Epoch 103
Validation binary_cross_entropy = 0.721613
Epoch 104
Loss = 4.9358e-01, PNorm = 64.9307, GNorm = 3.6929, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.539421
Epoch 105
Validation binary_cross_entropy = 0.292342
Epoch 106
Validation binary_cross_entropy = 0.315638
Epoch 107
Validation binary_cross_entropy = 0.412613
Epoch 108
Validation binary_cross_entropy = 0.486160
Epoch 109
Loss = 4.4880e-01, PNorm = 65.0490, GNorm = 1.6870, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.335090
Epoch 110
Validation binary_cross_entropy = 0.294233
Epoch 111
Validation binary_cross_entropy = 0.305402
Epoch 112
Validation binary_cross_entropy = 0.400336
Epoch 113
Validation binary_cross_entropy = 0.464611
Epoch 114
Loss = 3.8443e-01, PNorm = 65.1662, GNorm = 1.5615, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.391361
Epoch 115
Validation binary_cross_entropy = 0.396203
Epoch 116
Validation binary_cross_entropy = 0.513497
Epoch 117
Validation binary_cross_entropy = 0.418178
Epoch 118
Validation binary_cross_entropy = 0.388487
Epoch 119
Loss = 2.5380e-01, PNorm = 65.2913, GNorm = 1.3731, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.613354
Epoch 120
Validation binary_cross_entropy = 0.474538
Epoch 121
Validation binary_cross_entropy = 0.372315
Epoch 122
Validation binary_cross_entropy = 0.433803
Epoch 123
Validation binary_cross_entropy = 0.484063
Epoch 124
Loss = 3.0119e-01, PNorm = 65.4197, GNorm = 1.3540, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.401010
Epoch 125
Validation binary_cross_entropy = 0.420512
Epoch 126
Validation binary_cross_entropy = 0.483132
Epoch 127
Validation binary_cross_entropy = 0.374444
Epoch 128
Validation binary_cross_entropy = 0.332217
Epoch 129
Loss = 2.8078e-01, PNorm = 65.5430, GNorm = 1.7484, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.350676
Epoch 130
Validation binary_cross_entropy = 0.404997
Epoch 131
Validation binary_cross_entropy = 0.444125
Epoch 132
Validation binary_cross_entropy = 0.494462
Epoch 133
Validation binary_cross_entropy = 0.512761
Epoch 134
Loss = 3.1399e-01, PNorm = 65.6717, GNorm = 3.2445, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.539115
Epoch 135
Validation binary_cross_entropy = 0.498464
Epoch 136
Validation binary_cross_entropy = 0.431225
Epoch 137
Validation binary_cross_entropy = 0.433785
Epoch 138
Validation binary_cross_entropy = 0.529801
Epoch 139
Loss = 2.8763e-01, PNorm = 65.7962, GNorm = 2.5216, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.535909
Epoch 140
Validation binary_cross_entropy = 0.414471
Epoch 141
Validation binary_cross_entropy = 0.440450
Epoch 142
Validation binary_cross_entropy = 0.442936
Epoch 143
Validation binary_cross_entropy = 0.478012
Epoch 144
Loss = 2.9299e-01, PNorm = 65.9132, GNorm = 1.2119, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.443291
Epoch 145
Validation binary_cross_entropy = 0.427315
Epoch 146
Validation binary_cross_entropy = 0.428895
Epoch 147
Validation binary_cross_entropy = 0.439332
Epoch 148
Validation binary_cross_entropy = 0.470202
Epoch 149
Loss = 1.4828e-01, PNorm = 66.0316, GNorm = 1.2197, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.520475
Epoch 150
Validation binary_cross_entropy = 0.561264
Epoch 151
Validation binary_cross_entropy = 0.590115
Epoch 152
Validation binary_cross_entropy = 0.623507
Epoch 153
Validation binary_cross_entropy = 0.638113
Epoch 154
Loss = 1.7463e-01, PNorm = 66.1310, GNorm = 1.1854, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.613183
Epoch 155
Validation binary_cross_entropy = 0.607485
Epoch 156
Validation binary_cross_entropy = 0.623779
Epoch 157
Validation binary_cross_entropy = 0.591585
Epoch 158
Validation binary_cross_entropy = 0.534203
Epoch 159
Loss = 2.1993e-01, PNorm = 66.2261, GNorm = 1.2403, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.540053
Epoch 160
Validation binary_cross_entropy = 0.528178
Epoch 161
Validation binary_cross_entropy = 0.534948
Epoch 162
Validation binary_cross_entropy = 0.557417
Epoch 163
Validation binary_cross_entropy = 0.638133
Epoch 164
Loss = 2.6333e-01, PNorm = 66.3243, GNorm = 4.8232, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.555892
Epoch 165
Validation binary_cross_entropy = 0.514741
Epoch 166
Validation binary_cross_entropy = 0.492567
Epoch 167
Validation binary_cross_entropy = 0.512404
Epoch 168
Validation binary_cross_entropy = 0.558871
Epoch 169
Loss = 1.7197e-01, PNorm = 66.4210, GNorm = 2.6101, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.562796
Epoch 170
Validation binary_cross_entropy = 0.608897
Epoch 171
Validation binary_cross_entropy = 0.645343
Epoch 172
Validation binary_cross_entropy = 0.671981
Epoch 173
Validation binary_cross_entropy = 0.648177
Epoch 174
Loss = 1.5521e-01, PNorm = 66.5116, GNorm = 0.7514, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 0.655749
Epoch 175
Validation binary_cross_entropy = 0.665171
Epoch 176
Validation binary_cross_entropy = 0.688922
Epoch 177
Validation binary_cross_entropy = 0.709575
Epoch 178
Validation binary_cross_entropy = 0.716942
Epoch 179
Loss = 1.1609e-01, PNorm = 66.6013, GNorm = 0.9371, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 0.663592
Epoch 180
Validation binary_cross_entropy = 0.680874
Epoch 181
Validation binary_cross_entropy = 0.772537
Epoch 182
Validation binary_cross_entropy = 0.786908
Epoch 183
Validation binary_cross_entropy = 0.815885
Epoch 184
Loss = 9.7833e-02, PNorm = 66.6867, GNorm = 2.0965, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 0.829068
Epoch 185
Validation binary_cross_entropy = 0.782669
Epoch 186
Validation binary_cross_entropy = 0.758480
Epoch 187
Validation binary_cross_entropy = 0.709322
Epoch 188
Validation binary_cross_entropy = 0.703032
Epoch 189
Loss = 1.7394e-01, PNorm = 66.7703, GNorm = 3.2816, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 0.664423
Epoch 190
Validation binary_cross_entropy = 0.684821
Epoch 191
Validation binary_cross_entropy = 0.642785
Epoch 192
Validation binary_cross_entropy = 0.630588
Epoch 193
Validation binary_cross_entropy = 0.667887
Epoch 194
Loss = 1.6007e-01, PNorm = 66.8688, GNorm = 1.3725, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 0.760205
Epoch 195
Validation binary_cross_entropy = 0.686413
Epoch 196
Validation binary_cross_entropy = 0.668900
Epoch 197
Validation binary_cross_entropy = 0.672013
Epoch 198
Validation binary_cross_entropy = 0.723598
Epoch 199
Loss = 1.6143e-01, PNorm = 66.9533, GNorm = 1.6249, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 0.691179
Epoch 200
Validation binary_cross_entropy = 0.662706
Epoch 201
Validation binary_cross_entropy = 0.672972
Epoch 202
Validation binary_cross_entropy = 0.682454
Epoch 203
Validation binary_cross_entropy = 0.722743
Epoch 204
Loss = 1.0428e-01, PNorm = 67.0344, GNorm = 1.3031, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 0.801398
Epoch 205
Validation binary_cross_entropy = 0.842544
Epoch 206
Validation binary_cross_entropy = 0.849514
Epoch 207
Validation binary_cross_entropy = 0.890061
Epoch 208
Validation binary_cross_entropy = 0.914719
Epoch 209
Loss = 1.1801e-01, PNorm = 67.0975, GNorm = 4.6481, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 0.888746
Epoch 210
Validation binary_cross_entropy = 0.834803
Epoch 211
Validation binary_cross_entropy = 0.835002
Epoch 212
Validation binary_cross_entropy = 0.895135
Epoch 213
Validation binary_cross_entropy = 0.831179
Epoch 214
Loss = 1.0330e-01, PNorm = 67.1638, GNorm = 3.2304, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 0.716137
Epoch 215
Validation binary_cross_entropy = 0.673215
Epoch 216
Validation binary_cross_entropy = 0.685730
Epoch 217
Validation binary_cross_entropy = 0.789129
Epoch 218
Validation binary_cross_entropy = 0.789408
Epoch 219
Loss = 6.2000e-02, PNorm = 67.2472, GNorm = 0.5601, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 0.766084
Epoch 220
Validation binary_cross_entropy = 0.781463
Epoch 221
Validation binary_cross_entropy = 0.805680
Epoch 222
Validation binary_cross_entropy = 0.860668
Epoch 223
Validation binary_cross_entropy = 0.866936
Epoch 224
Loss = 8.2610e-02, PNorm = 67.3192, GNorm = 0.6525, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 0.868158
Epoch 225
Validation binary_cross_entropy = 0.854680
Epoch 226
Validation binary_cross_entropy = 0.852938
Epoch 227
Validation binary_cross_entropy = 0.875112
Epoch 228
Validation binary_cross_entropy = 0.890244
Epoch 229
Loss = 8.9978e-02, PNorm = 67.3817, GNorm = 1.5746, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 0.952227
Epoch 230
Validation binary_cross_entropy = 1.043060
Epoch 231
Validation binary_cross_entropy = 0.976974
Epoch 232
Validation binary_cross_entropy = 0.938446
Epoch 233
Validation binary_cross_entropy = 0.943160
Epoch 234
Loss = 7.4074e-02, PNorm = 67.4410, GNorm = 0.7492, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 0.959421
Epoch 235
Validation binary_cross_entropy = 0.958774
Epoch 236
Validation binary_cross_entropy = 0.969639
Epoch 237
Validation binary_cross_entropy = 1.002702
Epoch 238
Validation binary_cross_entropy = 0.959959
Epoch 239
Loss = 8.3489e-02, PNorm = 67.4944, GNorm = 2.3913, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 0.937509
Epoch 240
Validation binary_cross_entropy = 1.004860
Epoch 241
Validation binary_cross_entropy = 1.049602
Epoch 242
Validation binary_cross_entropy = 0.968815
Epoch 243
Validation binary_cross_entropy = 0.923934
Epoch 244
Loss = 6.5318e-02, PNorm = 67.5451, GNorm = 0.8162, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 0.849236
Epoch 245
Validation binary_cross_entropy = 0.839018
Epoch 246
Validation binary_cross_entropy = 0.783737
Epoch 247
Validation binary_cross_entropy = 0.737618
Epoch 248
Validation binary_cross_entropy = 0.742191
Epoch 249
Loss = 1.4853e-01, PNorm = 67.6103, GNorm = 1.6332, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 0.802002
Epoch 250
Validation binary_cross_entropy = 0.864917
Epoch 251
Validation binary_cross_entropy = 0.907121
Epoch 252
Validation binary_cross_entropy = 0.939002
Epoch 253
Validation binary_cross_entropy = 0.971051
Epoch 254
Loss = 4.7866e-02, PNorm = 67.6775, GNorm = 1.4719, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 1.015588
Epoch 255
Validation binary_cross_entropy = 1.068408
Epoch 256
Validation binary_cross_entropy = 1.134671
Epoch 257
Validation binary_cross_entropy = 1.128936
Epoch 258
Validation binary_cross_entropy = 1.092734
Epoch 259
Loss = 1.7808e-01, PNorm = 67.7298, GNorm = 1.2032, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 1.060179
Epoch 260
Validation binary_cross_entropy = 1.047304
Epoch 261
Validation binary_cross_entropy = 1.022848
Epoch 262
Validation binary_cross_entropy = 0.923475
Epoch 263
Validation binary_cross_entropy = 0.931697
Epoch 264
Loss = 2.0036e-01, PNorm = 67.7981, GNorm = 2.0497, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 0.831905
Epoch 265
Validation binary_cross_entropy = 0.845620
Epoch 266
Validation binary_cross_entropy = 0.843334
Epoch 267
Validation binary_cross_entropy = 0.744547
Epoch 268
Validation binary_cross_entropy = 0.699488
Epoch 269
Loss = 1.1784e-01, PNorm = 67.8713, GNorm = 2.3271, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 0.713096
Epoch 270
Validation binary_cross_entropy = 0.745419
Epoch 271
Validation binary_cross_entropy = 0.795106
Epoch 272
Validation binary_cross_entropy = 0.849127
Epoch 273
Validation binary_cross_entropy = 0.905021
Epoch 274
Loss = 5.1612e-02, PNorm = 67.9433, GNorm = 0.9625, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 0.913278
Epoch 275
Validation binary_cross_entropy = 0.957402
Epoch 276
Validation binary_cross_entropy = 0.996785
Epoch 277
Validation binary_cross_entropy = 1.122777
Epoch 278
Validation binary_cross_entropy = 1.158667
Epoch 279
Loss = 7.0621e-02, PNorm = 68.0026, GNorm = 3.4434, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 1.043021
Epoch 280
Validation binary_cross_entropy = 0.984485
Epoch 281
Validation binary_cross_entropy = 0.949563
Epoch 282
Validation binary_cross_entropy = 0.933258
Epoch 283
Validation binary_cross_entropy = 0.970764
Epoch 284
Loss = 1.0843e-01, PNorm = 68.0577, GNorm = 3.3861, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 0.949165
Epoch 285
Validation binary_cross_entropy = 0.904148
Epoch 286
Validation binary_cross_entropy = 0.869145
Epoch 287
Validation binary_cross_entropy = 0.875050
Epoch 288
Validation binary_cross_entropy = 0.931237
Epoch 289
Loss = 5.5501e-02, PNorm = 68.1115, GNorm = 1.0132, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 0.946712
Epoch 290
Validation binary_cross_entropy = 0.925204
Epoch 291
Validation binary_cross_entropy = 0.921000
Epoch 292
Validation binary_cross_entropy = 0.942452
Epoch 293
Validation binary_cross_entropy = 1.019716
Epoch 294
Loss = 8.9854e-02, PNorm = 68.1630, GNorm = 2.4913, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 1.182991
Epoch 295
Validation binary_cross_entropy = 0.987588
Epoch 296
Validation binary_cross_entropy = 0.935637
Epoch 297
Validation binary_cross_entropy = 0.931514
Epoch 298
Validation binary_cross_entropy = 0.905639
Epoch 299
Loss = 1.6295e-01, PNorm = 68.2377, GNorm = 1.3616, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 0.962049
Model 0 best validation binary_cross_entropy = 0.291358 on epoch 7
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.242493
Ensemble test binary_cross_entropy = 0.242493
Fold 1
Splitting data with seed 1
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.4, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=800, bias=False)
        (W_h): Linear(in_features=800, out_features=800, bias=False)
        (W_o): Linear(in_features=933, out_features=800, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.4, inplace=False)
    (1): Linear(in_features=3100, out_features=800, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.4, inplace=False)
    (4): Linear(in_features=800, out_features=800, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.4, inplace=False)
    (7): Linear(in_features=800, out_features=1, bias=True)
  )
)
Number of parameters = 4,627,201
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 0.289308
Epoch 1
Validation binary_cross_entropy = 0.308339
Epoch 2
Validation binary_cross_entropy = 0.745188
Epoch 3
Validation binary_cross_entropy = 0.310702
Epoch 4
Loss = 9.0577e-01, PNorm = 63.1501, GNorm = 10.4012, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 0.297426
Epoch 5
Validation binary_cross_entropy = 0.490699
Epoch 6
Validation binary_cross_entropy = 0.968219
Epoch 7
Validation binary_cross_entropy = 0.325563
Epoch 8
Validation binary_cross_entropy = 0.300375
Epoch 9
Loss = 7.2111e-01, PNorm = 63.2884, GNorm = 3.8343, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 0.761792
Epoch 10
Validation binary_cross_entropy = 0.812666
Epoch 11
Validation binary_cross_entropy = 0.304944
Epoch 12
Validation binary_cross_entropy = 0.370894
Epoch 13
Validation binary_cross_entropy = 0.966551
Epoch 14
Loss = 8.0977e-01, PNorm = 63.4289, GNorm = 5.2333, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 0.645028
Epoch 15
Validation binary_cross_entropy = 0.397379
Epoch 16
Validation binary_cross_entropy = 0.381264
Epoch 17
Validation binary_cross_entropy = 0.597862
Epoch 18
Validation binary_cross_entropy = 0.714151
Epoch 19
Loss = 6.9391e-01, PNorm = 63.5343, GNorm = 2.6725, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 0.397722
Epoch 20
Validation binary_cross_entropy = 0.372812
Epoch 21
Validation binary_cross_entropy = 0.542879
Epoch 22
Validation binary_cross_entropy = 0.724101
Epoch 23
Validation binary_cross_entropy = 0.487529
Epoch 24
Loss = 6.7108e-01, PNorm = 63.6206, GNorm = 3.4559, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 0.374869
Epoch 25
Validation binary_cross_entropy = 0.444442
Epoch 26
Validation binary_cross_entropy = 0.608225
Epoch 27
Validation binary_cross_entropy = 0.614403
Epoch 28
Validation binary_cross_entropy = 0.490906
Epoch 29
Loss = 6.3944e-01, PNorm = 63.6893, GNorm = 3.5720, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 0.468085
Epoch 30
Validation binary_cross_entropy = 0.588779
Epoch 31
Validation binary_cross_entropy = 0.543842
Epoch 32
Validation binary_cross_entropy = 0.345004
Epoch 33
Validation binary_cross_entropy = 0.393434
Epoch 34
Loss = 5.5787e-01, PNorm = 63.7513, GNorm = 2.3707, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 0.595307
Epoch 35
Validation binary_cross_entropy = 0.634573
Epoch 36
Validation binary_cross_entropy = 0.531896
Epoch 37
Validation binary_cross_entropy = 0.447408
Epoch 38
Validation binary_cross_entropy = 0.468821
Epoch 39
Loss = 6.9781e-01, PNorm = 63.8134, GNorm = 2.5434, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 0.568841
Epoch 40
Validation binary_cross_entropy = 0.593225
Epoch 41
Validation binary_cross_entropy = 0.540140
Epoch 42
Validation binary_cross_entropy = 0.507117
Epoch 43
Validation binary_cross_entropy = 0.587334
Epoch 44
Loss = 5.6295e-01, PNorm = 63.8690, GNorm = 3.0899, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 0.534266
Epoch 45
Validation binary_cross_entropy = 0.423658
Epoch 46
Validation binary_cross_entropy = 0.440844
Epoch 47
Validation binary_cross_entropy = 0.539909
Epoch 48
Validation binary_cross_entropy = 0.626085
Epoch 49
Loss = 6.2883e-01, PNorm = 63.9222, GNorm = 2.6025, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 0.597083
Epoch 50
Validation binary_cross_entropy = 0.487702
Epoch 51
Validation binary_cross_entropy = 0.489121
Epoch 52
Validation binary_cross_entropy = 0.583457
Epoch 53
Validation binary_cross_entropy = 0.522685
Epoch 54
Loss = 6.3797e-01, PNorm = 63.9804, GNorm = 2.1524, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 0.445645
Epoch 55
Validation binary_cross_entropy = 0.417933
Epoch 56
Validation binary_cross_entropy = 0.494369
Epoch 57
Validation binary_cross_entropy = 0.561727
Epoch 58
Validation binary_cross_entropy = 0.504752
Epoch 59
Loss = 5.6181e-01, PNorm = 64.0434, GNorm = 1.8386, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 0.511117
Epoch 60
Validation binary_cross_entropy = 0.512990
Epoch 61
Validation binary_cross_entropy = 0.410916
Epoch 62
Validation binary_cross_entropy = 0.425751
Epoch 63
Validation binary_cross_entropy = 0.399608
Epoch 64
Loss = 5.9481e-01, PNorm = 64.1133, GNorm = 1.7818, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 0.447627
Epoch 65
Validation binary_cross_entropy = 0.585543
Epoch 66
Validation binary_cross_entropy = 0.498073
Epoch 67
Validation binary_cross_entropy = 0.397328
Epoch 68
Validation binary_cross_entropy = 0.532170
Epoch 69
Loss = 5.3980e-01, PNorm = 64.1840, GNorm = 2.5068, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 0.617519
Epoch 70
Validation binary_cross_entropy = 0.357128
Epoch 71
Validation binary_cross_entropy = 0.362392
Epoch 72
Validation binary_cross_entropy = 0.513583
Epoch 73
Validation binary_cross_entropy = 0.645172
Epoch 74
Loss = 5.5613e-01, PNorm = 64.2727, GNorm = 1.7584, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 0.434225
Epoch 75
Validation binary_cross_entropy = 0.372060
Epoch 76
Validation binary_cross_entropy = 0.436520
Epoch 77
Validation binary_cross_entropy = 0.467176
Epoch 78
Validation binary_cross_entropy = 0.487819
Epoch 79
Loss = 4.6261e-01, PNorm = 64.3592, GNorm = 1.7598, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 0.466109
Epoch 80
Validation binary_cross_entropy = 0.632748
Epoch 81
Validation binary_cross_entropy = 0.433313
Epoch 82
Validation binary_cross_entropy = 0.361331
Epoch 83
Validation binary_cross_entropy = 0.537557
Epoch 84
Loss = 4.9723e-01, PNorm = 64.4520, GNorm = 2.1733, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 0.431467
Epoch 85
Validation binary_cross_entropy = 0.333543
Epoch 86
Validation binary_cross_entropy = 0.370174
Epoch 87
Validation binary_cross_entropy = 0.555276
Epoch 88
Validation binary_cross_entropy = 0.479054
Epoch 89
Loss = 4.5249e-01, PNorm = 64.5630, GNorm = 1.4847, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 0.329159
Epoch 90
Validation binary_cross_entropy = 0.380013
Epoch 91
Validation binary_cross_entropy = 0.409475
Epoch 92
Validation binary_cross_entropy = 0.364087
Epoch 93
Validation binary_cross_entropy = 0.476619
Epoch 94
Loss = 4.5156e-01, PNorm = 64.6781, GNorm = 1.3966, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 0.400531
Epoch 95
Validation binary_cross_entropy = 0.328011
Epoch 96
Validation binary_cross_entropy = 0.397281
Epoch 97
Validation binary_cross_entropy = 0.442510
Epoch 98
Validation binary_cross_entropy = 0.316867
Epoch 99
Loss = 3.9917e-01, PNorm = 64.7948, GNorm = 1.3384, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.381642
Epoch 100
Validation binary_cross_entropy = 0.340656
Epoch 101
Validation binary_cross_entropy = 0.343807
Epoch 102
Validation binary_cross_entropy = 0.409868
Epoch 103
Validation binary_cross_entropy = 0.331273
Epoch 104
Loss = 4.1234e-01, PNorm = 64.9178, GNorm = 2.1351, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.335633
Epoch 105
Validation binary_cross_entropy = 0.332484
Epoch 106
Validation binary_cross_entropy = 0.463478
Epoch 107
Validation binary_cross_entropy = 0.348700
Epoch 108
Validation binary_cross_entropy = 0.437580
Epoch 109
Loss = 3.7253e-01, PNorm = 65.0462, GNorm = 2.3925, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.392550
Epoch 110
Validation binary_cross_entropy = 0.364847
Epoch 111
Validation binary_cross_entropy = 0.437888
Epoch 112
Validation binary_cross_entropy = 0.386473
Epoch 113
Validation binary_cross_entropy = 0.712884
Epoch 114
Loss = 5.3870e-01, PNorm = 65.1742, GNorm = 4.7194, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.529560
Epoch 115
Validation binary_cross_entropy = 0.287942
Epoch 116
Validation binary_cross_entropy = 0.284855
Epoch 117
Validation binary_cross_entropy = 0.348772
Epoch 118
Validation binary_cross_entropy = 0.457036
Epoch 119
Loss = 4.3748e-01, PNorm = 65.3078, GNorm = 2.5577, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.349689
Epoch 120
Validation binary_cross_entropy = 0.314681
Epoch 121
Validation binary_cross_entropy = 0.337326
Epoch 122
Validation binary_cross_entropy = 0.412077
Epoch 123
Validation binary_cross_entropy = 0.388731
Epoch 124
Loss = 2.5967e-01, PNorm = 65.4271, GNorm = 1.8596, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.376822
Epoch 125
Validation binary_cross_entropy = 0.444676
Epoch 126
Validation binary_cross_entropy = 0.428776
Epoch 127
Validation binary_cross_entropy = 0.376223
Epoch 128
Validation binary_cross_entropy = 0.440184
Epoch 129
Loss = 3.0453e-01, PNorm = 65.5484, GNorm = 2.1337, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.404032
Epoch 130
Validation binary_cross_entropy = 0.382363
Epoch 131
Validation binary_cross_entropy = 0.387257
Epoch 132
Validation binary_cross_entropy = 0.453730
Epoch 133
Validation binary_cross_entropy = 0.388469
Epoch 134
Loss = 2.5706e-01, PNorm = 65.6563, GNorm = 3.7271, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.397460
Epoch 135
Validation binary_cross_entropy = 0.572592
Epoch 136
Validation binary_cross_entropy = 0.415203
Epoch 137
Validation binary_cross_entropy = 0.468749
Epoch 138
Validation binary_cross_entropy = 0.418509
Epoch 139
Loss = 2.2561e-01, PNorm = 65.7830, GNorm = 2.0765, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.517096
Epoch 140
Validation binary_cross_entropy = 0.434145
Epoch 141
Validation binary_cross_entropy = 0.436579
Epoch 142
Validation binary_cross_entropy = 0.438330
Epoch 143
Validation binary_cross_entropy = 0.451210
Epoch 144
Loss = 3.0741e-01, PNorm = 65.9027, GNorm = 1.8342, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.466161
Epoch 145
Validation binary_cross_entropy = 0.447815
Epoch 146
Validation binary_cross_entropy = 0.477796
Epoch 147
Validation binary_cross_entropy = 0.529831
Epoch 148
Validation binary_cross_entropy = 0.580780
Epoch 149
Loss = 2.0592e-01, PNorm = 66.0056, GNorm = 2.2377, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.587712
Epoch 150
Validation binary_cross_entropy = 0.592715
Epoch 151
Validation binary_cross_entropy = 0.583533
Epoch 152
Validation binary_cross_entropy = 0.536556
Epoch 153
Validation binary_cross_entropy = 0.515101
Epoch 154
Loss = 1.4351e-01, PNorm = 66.1022, GNorm = 1.7629, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.518773
Epoch 155
Validation binary_cross_entropy = 0.568467
Epoch 156
Validation binary_cross_entropy = 0.541344
Epoch 157
Validation binary_cross_entropy = 0.601563
Epoch 158
Validation binary_cross_entropy = 0.555374
Epoch 159
Loss = 2.1450e-01, PNorm = 66.2043, GNorm = 4.5361, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.572070
Epoch 160
Validation binary_cross_entropy = 0.543611
Epoch 161
Validation binary_cross_entropy = 0.550204
Epoch 162
Validation binary_cross_entropy = 0.446803
Epoch 163
Validation binary_cross_entropy = 0.501269
Epoch 164
Loss = 2.9160e-01, PNorm = 66.3242, GNorm = 4.4682, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.499625
Epoch 165
Validation binary_cross_entropy = 0.525995
Epoch 166
Validation binary_cross_entropy = 0.536545
Epoch 167
Validation binary_cross_entropy = 0.516530
Epoch 168
Validation binary_cross_entropy = 0.589170
Epoch 169
Loss = 2.2057e-01, PNorm = 66.4346, GNorm = 4.6080, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.590660
Epoch 170
Validation binary_cross_entropy = 0.562795
Epoch 171
Validation binary_cross_entropy = 0.546553
Epoch 172
Validation binary_cross_entropy = 0.573984
Epoch 173
Validation binary_cross_entropy = 0.605069
Epoch 174
Loss = 1.7019e-01, PNorm = 66.5300, GNorm = 1.2926, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 0.552804
Epoch 175
Validation binary_cross_entropy = 0.570557
Epoch 176
Validation binary_cross_entropy = 0.635902
Epoch 177
Validation binary_cross_entropy = 0.724224
Epoch 178
Validation binary_cross_entropy = 0.633745
Epoch 179
Loss = 2.8175e-01, PNorm = 66.6204, GNorm = 5.6135, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 0.553052
Epoch 180
Validation binary_cross_entropy = 0.524558
Epoch 181
Validation binary_cross_entropy = 0.545257
Epoch 182
Validation binary_cross_entropy = 0.465571
Epoch 183
Validation binary_cross_entropy = 0.468171
Epoch 184
Loss = 2.1440e-01, PNorm = 66.7146, GNorm = 1.2290, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 0.469993
Epoch 185
Validation binary_cross_entropy = 0.505197
Epoch 186
Validation binary_cross_entropy = 0.568072
Epoch 187
Validation binary_cross_entropy = 0.629627
Epoch 188
Validation binary_cross_entropy = 0.684553
Epoch 189
Loss = 2.8195e-01, PNorm = 66.8107, GNorm = 2.7529, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 0.707812
Epoch 190
Validation binary_cross_entropy = 0.688941
Epoch 191
Validation binary_cross_entropy = 0.684806
Epoch 192
Validation binary_cross_entropy = 0.663233
Epoch 193
Validation binary_cross_entropy = 0.645649
Epoch 194
Loss = 1.9146e-01, PNorm = 66.8932, GNorm = 3.0490, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 0.657359
Epoch 195
Validation binary_cross_entropy = 0.753763
Epoch 196
Validation binary_cross_entropy = 0.693986
Epoch 197
Validation binary_cross_entropy = 0.653661
Epoch 198
Validation binary_cross_entropy = 0.668118
Epoch 199
Loss = 1.1276e-01, PNorm = 66.9779, GNorm = 0.8511, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 0.694273
Epoch 200
Validation binary_cross_entropy = 0.756639
Epoch 201
Validation binary_cross_entropy = 0.799316
Epoch 202
Validation binary_cross_entropy = 0.719957
Epoch 203
Validation binary_cross_entropy = 0.793104
Epoch 204
Loss = 3.1371e-01, PNorm = 67.0476, GNorm = 3.4155, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 0.636993
Epoch 205
Validation binary_cross_entropy = 0.687180
Epoch 206
Validation binary_cross_entropy = 0.666155
Epoch 207
Validation binary_cross_entropy = 0.614224
Epoch 208
Validation binary_cross_entropy = 0.625573
Epoch 209
Loss = 1.3771e-01, PNorm = 67.1311, GNorm = 1.6311, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 0.673266
Epoch 210
Validation binary_cross_entropy = 0.760616
Epoch 211
Validation binary_cross_entropy = 0.844658
Epoch 212
Validation binary_cross_entropy = 0.758349
Epoch 213
Validation binary_cross_entropy = 0.674207
Epoch 214
Loss = 4.7765e-02, PNorm = 67.2067, GNorm = 0.3662, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 0.639954
Epoch 215
Validation binary_cross_entropy = 0.654130
Epoch 216
Validation binary_cross_entropy = 0.721411
Epoch 217
Validation binary_cross_entropy = 0.782650
Epoch 218
Validation binary_cross_entropy = 0.826679
Epoch 219
Loss = 1.8839e-01, PNorm = 67.2747, GNorm = 1.4270, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 0.826580
Epoch 220
Validation binary_cross_entropy = 0.857735
Epoch 221
Validation binary_cross_entropy = 0.909648
Epoch 222
Validation binary_cross_entropy = 0.921396
Epoch 223
Validation binary_cross_entropy = 0.859659
Epoch 224
Loss = 2.3139e-01, PNorm = 67.3357, GNorm = 3.3838, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 0.872573
Epoch 225
Validation binary_cross_entropy = 0.944735
Epoch 226
Validation binary_cross_entropy = 0.789036
Epoch 227
Validation binary_cross_entropy = 0.778644
Epoch 228
Validation binary_cross_entropy = 0.779214
Epoch 229
Loss = 6.5032e-02, PNorm = 67.3997, GNorm = 0.7788, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 0.818322
Epoch 230
Validation binary_cross_entropy = 0.835143
Epoch 231
Validation binary_cross_entropy = 0.828347
Epoch 232
Validation binary_cross_entropy = 0.844112
Epoch 233
Validation binary_cross_entropy = 0.858756
Epoch 234
Loss = 6.4748e-02, PNorm = 67.4588, GNorm = 1.2652, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 0.882232
Epoch 235
Validation binary_cross_entropy = 0.945686
Epoch 236
Validation binary_cross_entropy = 1.016312
Epoch 237
Validation binary_cross_entropy = 1.039401
Epoch 238
Validation binary_cross_entropy = 1.013759
Epoch 239
Loss = 7.7572e-02, PNorm = 67.5192, GNorm = 2.9065, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 1.006788
Epoch 240
Validation binary_cross_entropy = 1.060846
Epoch 241
Validation binary_cross_entropy = 1.133138
Epoch 242
Validation binary_cross_entropy = 0.855320
Epoch 243
Validation binary_cross_entropy = 0.743544
Epoch 244
Loss = 2.1591e-01, PNorm = 67.5761, GNorm = 2.7385, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 0.687889
Epoch 245
Validation binary_cross_entropy = 0.700435
Epoch 246
Validation binary_cross_entropy = 0.726985
Epoch 247
Validation binary_cross_entropy = 0.696845
Epoch 248
Validation binary_cross_entropy = 0.688697
Epoch 249
Loss = 1.3927e-01, PNorm = 67.6470, GNorm = 1.8450, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 0.716639
Epoch 250
Validation binary_cross_entropy = 0.759529
Epoch 251
Validation binary_cross_entropy = 0.839970
Epoch 252
Validation binary_cross_entropy = 0.912774
Epoch 253
Validation binary_cross_entropy = 0.968920
Epoch 254
Loss = 5.6834e-02, PNorm = 67.7127, GNorm = 1.4193, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 1.024158
Epoch 255
Validation binary_cross_entropy = 1.100507
Epoch 256
Validation binary_cross_entropy = 1.124451
Epoch 257
Validation binary_cross_entropy = 1.151587
Epoch 258
Validation binary_cross_entropy = 1.168363
Epoch 259
Loss = 2.1847e-02, PNorm = 67.7691, GNorm = 0.3959, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 1.170577
Epoch 260
Validation binary_cross_entropy = 1.150499
Epoch 261
Validation binary_cross_entropy = 1.137046
Epoch 262
Validation binary_cross_entropy = 1.142108
Epoch 263
Validation binary_cross_entropy = 1.081942
Epoch 264
Loss = 3.4890e-02, PNorm = 67.8215, GNorm = 1.0332, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 1.185257
Epoch 265
Validation binary_cross_entropy = 1.208048
Epoch 266
Validation binary_cross_entropy = 1.112380
Epoch 267
Validation binary_cross_entropy = 1.048141
Epoch 268
Validation binary_cross_entropy = 0.969296
Epoch 269
Loss = 4.3590e-02, PNorm = 67.8811, GNorm = 2.0567, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 0.966447
Epoch 270
Validation binary_cross_entropy = 0.920465
Epoch 271
Validation binary_cross_entropy = 0.913264
Epoch 272
Validation binary_cross_entropy = 1.067521
Epoch 273
Validation binary_cross_entropy = 1.069868
Epoch 274
Loss = 2.7767e-02, PNorm = 67.9443, GNorm = 0.3622, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 0.933185
Epoch 275
Validation binary_cross_entropy = 0.899960
Epoch 276
Validation binary_cross_entropy = 0.886046
Epoch 277
Validation binary_cross_entropy = 0.918143
Epoch 278
Validation binary_cross_entropy = 0.954700
Epoch 279
Loss = 8.0904e-02, PNorm = 68.0034, GNorm = 1.4508, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 0.899228
Epoch 280
Validation binary_cross_entropy = 0.874867
Epoch 281
Validation binary_cross_entropy = 0.897311
Epoch 282
Validation binary_cross_entropy = 0.940683
Epoch 283
Validation binary_cross_entropy = 1.050726
Epoch 284
Loss = 5.3572e-02, PNorm = 68.0619, GNorm = 1.6224, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 1.137102
Epoch 285
Validation binary_cross_entropy = 1.132738
Epoch 286
Validation binary_cross_entropy = 1.153462
Epoch 287
Validation binary_cross_entropy = 1.176990
Epoch 288
Validation binary_cross_entropy = 1.233647
Epoch 289
Loss = 2.9371e-02, PNorm = 68.1240, GNorm = 0.7905, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 1.296764
Epoch 290
Validation binary_cross_entropy = 1.281099
Epoch 291
Validation binary_cross_entropy = 1.198955
Epoch 292
Validation binary_cross_entropy = 1.186077
Epoch 293
Validation binary_cross_entropy = 1.252288
Epoch 294
Loss = 3.9203e-02, PNorm = 68.1847, GNorm = 1.4202, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 1.259666
Epoch 295
Validation binary_cross_entropy = 1.139686
Epoch 296
Validation binary_cross_entropy = 1.104261
Epoch 297
Validation binary_cross_entropy = 1.093130
Epoch 298
Validation binary_cross_entropy = 1.075757
Epoch 299
Loss = 4.9867e-02, PNorm = 68.2485, GNorm = 0.8541, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 1.134324
Model 0 best validation binary_cross_entropy = 0.284855 on epoch 116
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.198337
Ensemble test binary_cross_entropy = 0.198337
Fold 2
Splitting data with seed 2
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.4, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=800, bias=False)
        (W_h): Linear(in_features=800, out_features=800, bias=False)
        (W_o): Linear(in_features=933, out_features=800, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.4, inplace=False)
    (1): Linear(in_features=3100, out_features=800, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.4, inplace=False)
    (4): Linear(in_features=800, out_features=800, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.4, inplace=False)
    (7): Linear(in_features=800, out_features=1, bias=True)
  )
)
Number of parameters = 4,627,201
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 0.289617
Epoch 1
Validation binary_cross_entropy = 0.304917
Epoch 2
Validation binary_cross_entropy = 0.601288
Epoch 3
Validation binary_cross_entropy = 0.420122
Epoch 4
Loss = 7.6577e-01, PNorm = 63.1502, GNorm = 7.0863, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 0.297528
Epoch 5
Validation binary_cross_entropy = 0.422881
Epoch 6
Validation binary_cross_entropy = 0.756407
Epoch 7
Validation binary_cross_entropy = 0.303197
Epoch 8
Validation binary_cross_entropy = 0.332482
Epoch 9
Loss = 8.5060e-01, PNorm = 63.2876, GNorm = 5.5403, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 0.913284
Epoch 10
Validation binary_cross_entropy = 0.310293
Epoch 11
Validation binary_cross_entropy = 0.325268
Epoch 12
Validation binary_cross_entropy = 0.718814
Epoch 13
Validation binary_cross_entropy = 0.341210
Epoch 14
Loss = 8.1144e-01, PNorm = 63.4388, GNorm = 5.7765, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 0.346163
Epoch 15
Validation binary_cross_entropy = 0.693508
Epoch 16
Validation binary_cross_entropy = 0.361234
Epoch 17
Validation binary_cross_entropy = 0.303745
Epoch 18
Validation binary_cross_entropy = 0.468845
Epoch 19
Loss = 6.2844e-01, PNorm = 63.5597, GNorm = 5.5798, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 0.659127
Epoch 20
Validation binary_cross_entropy = 0.324386
Epoch 21
Validation binary_cross_entropy = 0.304827
Epoch 22
Validation binary_cross_entropy = 0.408427
Epoch 23
Validation binary_cross_entropy = 0.990125
Epoch 24
Loss = 7.3082e-01, PNorm = 63.6433, GNorm = 5.6986, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 0.674218
Epoch 25
Validation binary_cross_entropy = 0.367297
Epoch 26
Validation binary_cross_entropy = 0.418726
Epoch 27
Validation binary_cross_entropy = 0.720029
Epoch 28
Validation binary_cross_entropy = 0.713908
Epoch 29
Loss = 5.8398e-01, PNorm = 63.7123, GNorm = 2.9650, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 0.530297
Epoch 30
Validation binary_cross_entropy = 0.520336
Epoch 31
Validation binary_cross_entropy = 0.568638
Epoch 32
Validation binary_cross_entropy = 0.549944
Epoch 33
Validation binary_cross_entropy = 0.533363
Epoch 34
Loss = 6.9430e-01, PNorm = 63.7777, GNorm = 2.6115, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 0.541354
Epoch 35
Validation binary_cross_entropy = 0.560324
Epoch 36
Validation binary_cross_entropy = 0.444775
Epoch 37
Validation binary_cross_entropy = 0.415333
Epoch 38
Validation binary_cross_entropy = 0.591142
Epoch 39
Loss = 6.3802e-01, PNorm = 63.8303, GNorm = 3.3378, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 0.675578
Epoch 40
Validation binary_cross_entropy = 0.511834
Epoch 41
Validation binary_cross_entropy = 0.443027
Epoch 42
Validation binary_cross_entropy = 0.516000
Epoch 43
Validation binary_cross_entropy = 0.583309
Epoch 44
Loss = 6.6222e-01, PNorm = 63.8797, GNorm = 3.1191, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 0.601022
Epoch 45
Validation binary_cross_entropy = 0.431168
Epoch 46
Validation binary_cross_entropy = 0.402526
Epoch 47
Validation binary_cross_entropy = 0.422764
Epoch 48
Validation binary_cross_entropy = 0.484612
Epoch 49
Loss = 6.1539e-01, PNorm = 63.9383, GNorm = 2.3557, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 0.517515
Epoch 50
Validation binary_cross_entropy = 0.467650
Epoch 51
Validation binary_cross_entropy = 0.433795
Epoch 52
Validation binary_cross_entropy = 0.488805
Epoch 53
Validation binary_cross_entropy = 0.514093
Epoch 54
Loss = 5.8996e-01, PNorm = 63.9949, GNorm = 2.0851, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 0.531979
Epoch 55
Validation binary_cross_entropy = 0.505982
Epoch 56
Validation binary_cross_entropy = 0.417945
Epoch 57
Validation binary_cross_entropy = 0.415940
Epoch 58
Validation binary_cross_entropy = 0.375466
Epoch 59
Loss = 6.2153e-01, PNorm = 64.0547, GNorm = 1.6428, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 0.461701
Epoch 60
Validation binary_cross_entropy = 0.625850
Epoch 61
Validation binary_cross_entropy = 0.607951
Epoch 62
Validation binary_cross_entropy = 0.487565
Epoch 63
Validation binary_cross_entropy = 0.463475
Epoch 64
Loss = 5.3557e-01, PNorm = 64.1314, GNorm = 2.1343, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 0.475087
Epoch 65
Validation binary_cross_entropy = 0.396542
Epoch 66
Validation binary_cross_entropy = 0.415175
Epoch 67
Validation binary_cross_entropy = 0.509714
Epoch 68
Validation binary_cross_entropy = 0.438522
Epoch 69
Loss = 5.2263e-01, PNorm = 64.2227, GNorm = 1.5761, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 0.438391
Epoch 70
Validation binary_cross_entropy = 0.499057
Epoch 71
Validation binary_cross_entropy = 0.364971
Epoch 72
Validation binary_cross_entropy = 0.558006
Epoch 73
Validation binary_cross_entropy = 0.470181
Epoch 74
Loss = 5.0133e-01, PNorm = 64.3167, GNorm = 1.3991, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 0.387823
Epoch 75
Validation binary_cross_entropy = 0.368294
Epoch 76
Validation binary_cross_entropy = 0.414867
Epoch 77
Validation binary_cross_entropy = 0.493050
Epoch 78
Validation binary_cross_entropy = 0.328329
Epoch 79
Loss = 5.2386e-01, PNorm = 64.4258, GNorm = 1.7192, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 0.390897
Epoch 80
Validation binary_cross_entropy = 0.488785
Epoch 81
Validation binary_cross_entropy = 0.270032
Epoch 82
Validation binary_cross_entropy = 0.436148
Epoch 83
Validation binary_cross_entropy = 0.542536
Epoch 84
Loss = 5.1344e-01, PNorm = 64.5311, GNorm = 1.3049, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 0.401678
Epoch 85
Validation binary_cross_entropy = 0.380955
Epoch 86
Validation binary_cross_entropy = 0.418707
Epoch 87
Validation binary_cross_entropy = 0.469891
Epoch 88
Validation binary_cross_entropy = 0.428597
Epoch 89
Loss = 5.1390e-01, PNorm = 64.6448, GNorm = 1.6964, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 0.346294
Epoch 90
Validation binary_cross_entropy = 0.355845
Epoch 91
Validation binary_cross_entropy = 0.488675
Epoch 92
Validation binary_cross_entropy = 0.337905
Epoch 93
Validation binary_cross_entropy = 0.352543
Epoch 94
Loss = 3.7831e-01, PNorm = 64.7592, GNorm = 1.8561, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 0.339665
Epoch 95
Validation binary_cross_entropy = 0.359025
Epoch 96
Validation binary_cross_entropy = 0.303552
Epoch 97
Validation binary_cross_entropy = 0.354828
Epoch 98
Validation binary_cross_entropy = 0.390987
Epoch 99
Loss = 3.7516e-01, PNorm = 64.8644, GNorm = 2.0720, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.314549
Epoch 100
Validation binary_cross_entropy = 0.422985
Epoch 101
Validation binary_cross_entropy = 0.389920
Epoch 102
Validation binary_cross_entropy = 0.365315
Epoch 103
Validation binary_cross_entropy = 0.429874
Epoch 104
Loss = 4.3461e-01, PNorm = 64.9902, GNorm = 1.4622, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.309185
Epoch 105
Validation binary_cross_entropy = 0.398383
Epoch 106
Validation binary_cross_entropy = 0.423090
Epoch 107
Validation binary_cross_entropy = 0.304349
Epoch 108
Validation binary_cross_entropy = 0.327756
Epoch 109
Loss = 3.9548e-01, PNorm = 65.1263, GNorm = 2.6429, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.373369
Epoch 110
Validation binary_cross_entropy = 0.360013
Epoch 111
Validation binary_cross_entropy = 0.585442
Epoch 112
Validation binary_cross_entropy = 0.465499
Epoch 113
Validation binary_cross_entropy = 0.406105
Epoch 114
Loss = 2.9206e-01, PNorm = 65.2615, GNorm = 1.6956, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.341595
Epoch 115
Validation binary_cross_entropy = 0.342887
Epoch 116
Validation binary_cross_entropy = 0.409773
Epoch 117
Validation binary_cross_entropy = 0.348576
Epoch 118
Validation binary_cross_entropy = 0.447500
Epoch 119
Loss = 3.0793e-01, PNorm = 65.4015, GNorm = 2.2321, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.439217
Epoch 120
Validation binary_cross_entropy = 0.368593
Epoch 121
Validation binary_cross_entropy = 0.426201
Epoch 122
Validation binary_cross_entropy = 0.394298
Epoch 123
Validation binary_cross_entropy = 0.365323
Epoch 124
Loss = 1.8249e-01, PNorm = 65.5519, GNorm = 1.0571, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.482644
Epoch 125
Validation binary_cross_entropy = 0.548106
Epoch 126
Validation binary_cross_entropy = 0.406232
Epoch 127
Validation binary_cross_entropy = 0.504324
Epoch 128
Validation binary_cross_entropy = 0.385373
Epoch 129
Loss = 3.2431e-01, PNorm = 65.6953, GNorm = 4.5602, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.474217
Epoch 130
Validation binary_cross_entropy = 0.518473
Epoch 131
Validation binary_cross_entropy = 0.352604
Epoch 132
Validation binary_cross_entropy = 0.368539
Epoch 133
Validation binary_cross_entropy = 0.583950
Epoch 134
Loss = 2.4005e-01, PNorm = 65.8387, GNorm = 1.7376, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.476216
Epoch 135
Validation binary_cross_entropy = 0.413791
Epoch 136
Validation binary_cross_entropy = 0.412420
Epoch 137
Validation binary_cross_entropy = 0.433675
Epoch 138
Validation binary_cross_entropy = 0.467763
Epoch 139
Loss = 3.2667e-01, PNorm = 65.9711, GNorm = 1.2877, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.376226
Epoch 140
Validation binary_cross_entropy = 0.396020
Epoch 141
Validation binary_cross_entropy = 0.548683
Epoch 142
Validation binary_cross_entropy = 0.477305
Epoch 143
Validation binary_cross_entropy = 0.432081
Epoch 144
Loss = 1.8912e-01, PNorm = 66.1027, GNorm = 2.6414, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.447471
Epoch 145
Validation binary_cross_entropy = 0.506178
Epoch 146
Validation binary_cross_entropy = 0.505655
Epoch 147
Validation binary_cross_entropy = 0.499793
Epoch 148
Validation binary_cross_entropy = 0.501598
Epoch 149
Loss = 1.8092e-01, PNorm = 66.2169, GNorm = 1.1772, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.529852
Epoch 150
Validation binary_cross_entropy = 0.546075
Epoch 151
Validation binary_cross_entropy = 0.540012
Epoch 152
Validation binary_cross_entropy = 0.537561
Epoch 153
Validation binary_cross_entropy = 0.575755
Epoch 154
Loss = 2.2818e-01, PNorm = 66.3099, GNorm = 1.2615, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.528691
Epoch 155
Validation binary_cross_entropy = 0.535657
Epoch 156
Validation binary_cross_entropy = 0.569985
Epoch 157
Validation binary_cross_entropy = 0.612173
Epoch 158
Validation binary_cross_entropy = 0.560559
Epoch 159
Loss = 2.5113e-01, PNorm = 66.4037, GNorm = 5.9152, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.550427
Epoch 160
Validation binary_cross_entropy = 0.531274
Epoch 161
Validation binary_cross_entropy = 0.502541
Epoch 162
Validation binary_cross_entropy = 0.477752
Epoch 163
Validation binary_cross_entropy = 0.479849
Epoch 164
Loss = 1.7858e-01, PNorm = 66.5064, GNorm = 2.3603, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.476712
Epoch 165
Validation binary_cross_entropy = 0.492389
Epoch 166
Validation binary_cross_entropy = 0.511867
Epoch 167
Validation binary_cross_entropy = 0.559460
Epoch 168
Validation binary_cross_entropy = 0.568172
Epoch 169
Loss = 2.3865e-01, PNorm = 66.6028, GNorm = 2.5571, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.573370
Epoch 170
Validation binary_cross_entropy = 0.582988
Epoch 171
Validation binary_cross_entropy = 0.611908
Epoch 172
Validation binary_cross_entropy = 0.662162
Epoch 173
Validation binary_cross_entropy = 0.648682
Epoch 174
Loss = 1.1630e-01, PNorm = 66.6882, GNorm = 2.2408, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 0.660404
Epoch 175
Validation binary_cross_entropy = 0.663350
Epoch 176
Validation binary_cross_entropy = 0.688907
Epoch 177
Validation binary_cross_entropy = 0.699619
Epoch 178
Validation binary_cross_entropy = 0.737166
Epoch 179
Loss = 1.7644e-01, PNorm = 66.7722, GNorm = 1.9250, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 0.753317
Epoch 180
Validation binary_cross_entropy = 0.752408
Epoch 181
Validation binary_cross_entropy = 0.727404
Epoch 182
Validation binary_cross_entropy = 0.722525
Epoch 183
Validation binary_cross_entropy = 0.697629
Epoch 184
Loss = 1.6365e-01, PNorm = 66.8553, GNorm = 0.9590, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 0.704325
Epoch 185
Validation binary_cross_entropy = 0.684196
Epoch 186
Validation binary_cross_entropy = 0.661657
Epoch 187
Validation binary_cross_entropy = 0.680440
Epoch 188
Validation binary_cross_entropy = 0.660377
Epoch 189
Loss = 1.4425e-01, PNorm = 66.9331, GNorm = 2.3267, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 0.660017
Epoch 190
Validation binary_cross_entropy = 0.669937
Epoch 191
Validation binary_cross_entropy = 0.702215
Epoch 192
Validation binary_cross_entropy = 0.741260
Epoch 193
Validation binary_cross_entropy = 0.767917
Epoch 194
Loss = 9.1339e-02, PNorm = 67.0233, GNorm = 2.3180, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 0.801477
Epoch 195
Validation binary_cross_entropy = 0.857411
Epoch 196
Validation binary_cross_entropy = 0.707694
Epoch 197
Validation binary_cross_entropy = 0.659324
Epoch 198
Validation binary_cross_entropy = 0.619669
Epoch 199
Loss = 1.4107e-01, PNorm = 67.1039, GNorm = 1.0598, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 0.607717
Epoch 200
Validation binary_cross_entropy = 0.597727
Epoch 201
Validation binary_cross_entropy = 0.631146
Epoch 202
Validation binary_cross_entropy = 0.660322
Epoch 203
Validation binary_cross_entropy = 0.783280
Epoch 204
Loss = 1.8524e-01, PNorm = 67.1870, GNorm = 4.2221, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 0.797089
Epoch 205
Validation binary_cross_entropy = 0.795312
Epoch 206
Validation binary_cross_entropy = 0.851332
Epoch 207
Validation binary_cross_entropy = 0.918376
Epoch 208
Validation binary_cross_entropy = 0.792847
Epoch 209
Loss = 4.6530e-02, PNorm = 67.2745, GNorm = 0.6507, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 0.751124
Epoch 210
Validation binary_cross_entropy = 0.682671
Epoch 211
Validation binary_cross_entropy = 0.642507
Epoch 212
Validation binary_cross_entropy = 0.636322
Epoch 213
Validation binary_cross_entropy = 0.634993
Epoch 214
Loss = 1.8258e-01, PNorm = 67.3645, GNorm = 1.3987, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 0.644929
Epoch 215
Validation binary_cross_entropy = 0.612446
Epoch 216
Validation binary_cross_entropy = 0.651779
Epoch 217
Validation binary_cross_entropy = 0.687839
Epoch 218
Validation binary_cross_entropy = 0.687274
Epoch 219
Loss = 1.3855e-01, PNorm = 67.4513, GNorm = 1.8090, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 0.788704
Epoch 220
Validation binary_cross_entropy = 0.675384
Epoch 221
Validation binary_cross_entropy = 0.712676
Epoch 222
Validation binary_cross_entropy = 0.668932
Epoch 223
Validation binary_cross_entropy = 0.587459
Epoch 224
Loss = 9.2291e-02, PNorm = 67.5186, GNorm = 0.8868, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 0.575271
Epoch 225
Validation binary_cross_entropy = 0.580232
Epoch 226
Validation binary_cross_entropy = 0.606595
Epoch 227
Validation binary_cross_entropy = 0.668242
Epoch 228
Validation binary_cross_entropy = 0.694999
Epoch 229
Loss = 8.1139e-02, PNorm = 67.5856, GNorm = 1.0513, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 0.729251
Epoch 230
Validation binary_cross_entropy = 0.772452
Epoch 231
Validation binary_cross_entropy = 0.809136
Epoch 232
Validation binary_cross_entropy = 0.830582
Epoch 233
Validation binary_cross_entropy = 0.828586
Epoch 234
Loss = 5.7201e-02, PNorm = 67.6479, GNorm = 2.1207, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 0.850209
Epoch 235
Validation binary_cross_entropy = 0.885590
Epoch 236
Validation binary_cross_entropy = 0.929206
Epoch 237
Validation binary_cross_entropy = 0.936283
Epoch 238
Validation binary_cross_entropy = 0.953206
Epoch 239
Loss = 1.3627e-01, PNorm = 67.7015, GNorm = 4.9084, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 0.946843
Epoch 240
Validation binary_cross_entropy = 0.904491
Epoch 241
Validation binary_cross_entropy = 0.864527
Epoch 242
Validation binary_cross_entropy = 0.836601
Epoch 243
Validation binary_cross_entropy = 0.824363
Epoch 244
Loss = 1.0365e-01, PNorm = 67.7502, GNorm = 1.5948, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 0.816309
Epoch 245
Validation binary_cross_entropy = 0.805920
Epoch 246
Validation binary_cross_entropy = 0.816564
Epoch 247
Validation binary_cross_entropy = 0.832855
Epoch 248
Validation binary_cross_entropy = 0.839924
Epoch 249
Loss = 9.9414e-02, PNorm = 67.8105, GNorm = 0.8862, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 0.858812
Epoch 250
Validation binary_cross_entropy = 0.884752
Epoch 251
Validation binary_cross_entropy = 0.943205
Epoch 252
Validation binary_cross_entropy = 0.871087
Epoch 253
Validation binary_cross_entropy = 0.843792
Epoch 254
Loss = 5.8441e-02, PNorm = 67.8776, GNorm = 1.2369, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 0.834478
Epoch 255
Validation binary_cross_entropy = 0.823732
Epoch 256
Validation binary_cross_entropy = 0.827302
Epoch 257
Validation binary_cross_entropy = 0.841992
Epoch 258
Validation binary_cross_entropy = 0.844975
Epoch 259
Loss = 3.9627e-02, PNorm = 67.9412, GNorm = 1.5323, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 0.861016
Epoch 260
Validation binary_cross_entropy = 0.891484
Epoch 261
Validation binary_cross_entropy = 0.930675
Epoch 262
Validation binary_cross_entropy = 0.976460
Epoch 263
Validation binary_cross_entropy = 1.013079
Epoch 264
Loss = 2.5555e-02, PNorm = 67.9924, GNorm = 0.7473, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 1.048298
Epoch 265
Validation binary_cross_entropy = 1.085888
Epoch 266
Validation binary_cross_entropy = 1.136242
Epoch 267
Validation binary_cross_entropy = 1.138663
Epoch 268
Validation binary_cross_entropy = 1.130452
Epoch 269
Loss = 9.2432e-02, PNorm = 68.0279, GNorm = 0.8972, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 1.149886
Epoch 270
Validation binary_cross_entropy = 1.094458
Epoch 271
Validation binary_cross_entropy = 1.043505
Epoch 272
Validation binary_cross_entropy = 1.029505
Epoch 273
Validation binary_cross_entropy = 1.020263
Epoch 274
Loss = 5.9578e-02, PNorm = 68.0705, GNorm = 2.8084, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 1.024821
Epoch 275
Validation binary_cross_entropy = 1.015292
Epoch 276
Validation binary_cross_entropy = 1.076390
Epoch 277
Validation binary_cross_entropy = 1.029383
Epoch 278
Validation binary_cross_entropy = 0.931234
Epoch 279
Loss = 7.1647e-02, PNorm = 68.1198, GNorm = 1.1033, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 0.881514
Epoch 280
Validation binary_cross_entropy = 0.879637
Epoch 281
Validation binary_cross_entropy = 0.899193
Epoch 282
Validation binary_cross_entropy = 0.932059
Epoch 283
Validation binary_cross_entropy = 1.008843
Epoch 284
Loss = 8.2893e-02, PNorm = 68.1745, GNorm = 1.4582, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 1.105471
Epoch 285
Validation binary_cross_entropy = 1.096739
Epoch 286
Validation binary_cross_entropy = 1.113582
Epoch 287
Validation binary_cross_entropy = 1.121947
Epoch 288
Validation binary_cross_entropy = 1.177566
Epoch 289
Loss = 2.7953e-02, PNorm = 68.2217, GNorm = 2.0306, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 1.314124
Epoch 290
Validation binary_cross_entropy = 1.178311
Epoch 291
Validation binary_cross_entropy = 1.073959
Epoch 292
Validation binary_cross_entropy = 1.002623
Epoch 293
Validation binary_cross_entropy = 0.782546
Epoch 294
Loss = 2.4711e-01, PNorm = 68.2811, GNorm = 1.2912, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 0.702418
Epoch 295
Validation binary_cross_entropy = 0.641909
Epoch 296
Validation binary_cross_entropy = 0.570542
Epoch 297
Validation binary_cross_entropy = 0.546948
Epoch 298
Validation binary_cross_entropy = 0.553379
Epoch 299
Loss = 1.2045e-01, PNorm = 68.3430, GNorm = 0.9221, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 0.574303
Model 0 best validation binary_cross_entropy = 0.270032 on epoch 81
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.205699
Ensemble test binary_cross_entropy = 0.205699
Fold 3
Splitting data with seed 3
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.4, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=800, bias=False)
        (W_h): Linear(in_features=800, out_features=800, bias=False)
        (W_o): Linear(in_features=933, out_features=800, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.4, inplace=False)
    (1): Linear(in_features=3100, out_features=800, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.4, inplace=False)
    (4): Linear(in_features=800, out_features=800, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.4, inplace=False)
    (7): Linear(in_features=800, out_features=1, bias=True)
  )
)
Number of parameters = 4,627,201
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 0.292552
Epoch 1
Validation binary_cross_entropy = 0.314568
Epoch 2
Validation binary_cross_entropy = 0.721326
Epoch 3
Validation binary_cross_entropy = 0.305099
Epoch 4
Loss = 8.1420e-01, PNorm = 63.1529, GNorm = 6.9891, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 0.290179
Epoch 5
Validation binary_cross_entropy = 0.581570
Epoch 6
Validation binary_cross_entropy = 0.592783
Epoch 7
Validation binary_cross_entropy = 0.288288
Epoch 8
Validation binary_cross_entropy = 0.323829
Epoch 9
Loss = 6.8788e-01, PNorm = 63.2945, GNorm = 4.8888, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 0.737028
Epoch 10
Validation binary_cross_entropy = 0.332306
Epoch 11
Validation binary_cross_entropy = 0.324212
Epoch 12
Validation binary_cross_entropy = 0.496599
Epoch 13
Validation binary_cross_entropy = 0.373763
Epoch 14
Loss = 7.4570e-01, PNorm = 63.4460, GNorm = 4.5970, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 0.318944
Epoch 15
Validation binary_cross_entropy = 0.560535
Epoch 16
Validation binary_cross_entropy = 0.523742
Epoch 17
Validation binary_cross_entropy = 0.319813
Epoch 18
Validation binary_cross_entropy = 0.371191
Epoch 19
Loss = 7.1945e-01, PNorm = 63.5611, GNorm = 3.8456, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 0.643111
Epoch 20
Validation binary_cross_entropy = 0.409281
Epoch 21
Validation binary_cross_entropy = 0.307546
Epoch 22
Validation binary_cross_entropy = 0.365077
Epoch 23
Validation binary_cross_entropy = 0.621935
Epoch 24
Loss = 6.8321e-01, PNorm = 63.6446, GNorm = 4.3268, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 0.424910
Epoch 25
Validation binary_cross_entropy = 0.333438
Epoch 26
Validation binary_cross_entropy = 0.416331
Epoch 27
Validation binary_cross_entropy = 0.688589
Epoch 28
Validation binary_cross_entropy = 0.508631
Epoch 29
Loss = 5.8375e-01, PNorm = 63.7217, GNorm = 2.2870, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 0.449368
Epoch 30
Validation binary_cross_entropy = 0.498492
Epoch 31
Validation binary_cross_entropy = 0.588410
Epoch 32
Validation binary_cross_entropy = 0.406770
Epoch 33
Validation binary_cross_entropy = 0.467535
Epoch 34
Loss = 6.7340e-01, PNorm = 63.7850, GNorm = 2.9855, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 0.831251
Epoch 35
Validation binary_cross_entropy = 0.867948
Epoch 36
Validation binary_cross_entropy = 0.526878
Epoch 37
Validation binary_cross_entropy = 0.382611
Epoch 38
Validation binary_cross_entropy = 0.437268
Epoch 39
Loss = 6.4439e-01, PNorm = 63.8663, GNorm = 2.9600, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 0.646669
Epoch 40
Validation binary_cross_entropy = 0.688491
Epoch 41
Validation binary_cross_entropy = 0.555366
Epoch 42
Validation binary_cross_entropy = 0.426560
Epoch 43
Validation binary_cross_entropy = 0.391114
Epoch 44
Loss = 6.4086e-01, PNorm = 63.9220, GNorm = 2.3617, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 0.460353
Epoch 45
Validation binary_cross_entropy = 0.567913
Epoch 46
Validation binary_cross_entropy = 0.612058
Epoch 47
Validation binary_cross_entropy = 0.529652
Epoch 48
Validation binary_cross_entropy = 0.482848
Epoch 49
Loss = 5.5242e-01, PNorm = 63.9805, GNorm = 1.8685, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 0.466197
Epoch 50
Validation binary_cross_entropy = 0.488856
Epoch 51
Validation binary_cross_entropy = 0.517385
Epoch 52
Validation binary_cross_entropy = 0.512792
Epoch 53
Validation binary_cross_entropy = 0.483654
Epoch 54
Loss = 5.3956e-01, PNorm = 64.0228, GNorm = 1.8113, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 0.495857
Epoch 55
Validation binary_cross_entropy = 0.578116
Epoch 56
Validation binary_cross_entropy = 0.611870
Epoch 57
Validation binary_cross_entropy = 0.549744
Epoch 58
Validation binary_cross_entropy = 0.485417
Epoch 59
Loss = 6.3925e-01, PNorm = 64.0839, GNorm = 1.9920, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 0.446963
Epoch 60
Validation binary_cross_entropy = 0.538598
Epoch 61
Validation binary_cross_entropy = 0.583346
Epoch 62
Validation binary_cross_entropy = 0.561089
Epoch 63
Validation binary_cross_entropy = 0.496523
Epoch 64
Loss = 5.6670e-01, PNorm = 64.1440, GNorm = 1.5366, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 0.472133
Epoch 65
Validation binary_cross_entropy = 0.485509
Epoch 66
Validation binary_cross_entropy = 0.524734
Epoch 67
Validation binary_cross_entropy = 0.580696
Epoch 68
Validation binary_cross_entropy = 0.499647
Epoch 69
Loss = 5.4936e-01, PNorm = 64.2028, GNorm = 1.5289, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 0.460163
Epoch 70
Validation binary_cross_entropy = 0.507534
Epoch 71
Validation binary_cross_entropy = 0.564477
Epoch 72
Validation binary_cross_entropy = 0.439626
Epoch 73
Validation binary_cross_entropy = 0.401142
Epoch 74
Loss = 5.3582e-01, PNorm = 64.2652, GNorm = 1.6479, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 0.489256
Epoch 75
Validation binary_cross_entropy = 0.612832
Epoch 76
Validation binary_cross_entropy = 0.634315
Epoch 77
Validation binary_cross_entropy = 0.465726
Epoch 78
Validation binary_cross_entropy = 0.378660
Epoch 79
Loss = 5.7250e-01, PNorm = 64.3428, GNorm = 2.6554, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 0.432455
Epoch 80
Validation binary_cross_entropy = 0.528566
Epoch 81
Validation binary_cross_entropy = 0.494160
Epoch 82
Validation binary_cross_entropy = 0.389119
Epoch 83
Validation binary_cross_entropy = 0.386911
Epoch 84
Loss = 4.2573e-01, PNorm = 64.4444, GNorm = 1.8711, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 0.626095
Epoch 85
Validation binary_cross_entropy = 0.357479
Epoch 86
Validation binary_cross_entropy = 0.468645
Epoch 87
Validation binary_cross_entropy = 0.396168
Epoch 88
Validation binary_cross_entropy = 0.541328
Epoch 89
Loss = 5.1210e-01, PNorm = 64.5409, GNorm = 2.1628, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 0.432208
Epoch 90
Validation binary_cross_entropy = 0.350868
Epoch 91
Validation binary_cross_entropy = 0.405249
Epoch 92
Validation binary_cross_entropy = 0.386058
Epoch 93
Validation binary_cross_entropy = 0.354566
Epoch 94
Loss = 4.3531e-01, PNorm = 64.6587, GNorm = 1.8769, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 0.406523
Epoch 95
Validation binary_cross_entropy = 0.481656
Epoch 96
Validation binary_cross_entropy = 0.307152
Epoch 97
Validation binary_cross_entropy = 0.415882
Epoch 98
Validation binary_cross_entropy = 0.470450
Epoch 99
Loss = 3.9036e-01, PNorm = 64.7764, GNorm = 1.3740, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.388397
Epoch 100
Validation binary_cross_entropy = 0.388972
Epoch 101
Validation binary_cross_entropy = 0.426312
Epoch 102
Validation binary_cross_entropy = 0.335682
Epoch 103
Validation binary_cross_entropy = 0.362589
Epoch 104
Loss = 4.7210e-01, PNorm = 64.9009, GNorm = 2.5020, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.367900
Epoch 105
Validation binary_cross_entropy = 0.415055
Epoch 106
Validation binary_cross_entropy = 0.353359
Epoch 107
Validation binary_cross_entropy = 0.378082
Epoch 108
Validation binary_cross_entropy = 0.394396
Epoch 109
Loss = 4.9544e-01, PNorm = 65.0290, GNorm = 4.3292, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.327714
Epoch 110
Validation binary_cross_entropy = 0.533865
Epoch 111
Validation binary_cross_entropy = 0.397293
Epoch 112
Validation binary_cross_entropy = 0.339235
Epoch 113
Validation binary_cross_entropy = 0.558443
Epoch 114
Loss = 5.1693e-01, PNorm = 65.1670, GNorm = 2.9845, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.440673
Epoch 115
Validation binary_cross_entropy = 0.307551
Epoch 116
Validation binary_cross_entropy = 0.482742
Epoch 117
Validation binary_cross_entropy = 0.441121
Epoch 118
Validation binary_cross_entropy = 0.349394
Epoch 119
Loss = 3.4035e-01, PNorm = 65.3126, GNorm = 1.3693, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.366206
Epoch 120
Validation binary_cross_entropy = 0.379846
Epoch 121
Validation binary_cross_entropy = 0.405693
Epoch 122
Validation binary_cross_entropy = 0.413864
Epoch 123
Validation binary_cross_entropy = 0.425533
Epoch 124
Loss = 3.2348e-01, PNorm = 65.4496, GNorm = 2.9674, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.547722
Epoch 125
Validation binary_cross_entropy = 0.355873
Epoch 126
Validation binary_cross_entropy = 0.392602
Epoch 127
Validation binary_cross_entropy = 0.661419
Epoch 128
Validation binary_cross_entropy = 0.368440
Epoch 129
Loss = 3.6698e-01, PNorm = 65.5949, GNorm = 1.4752, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.352750
Epoch 130
Validation binary_cross_entropy = 0.411927
Epoch 131
Validation binary_cross_entropy = 0.481080
Epoch 132
Validation binary_cross_entropy = 0.402232
Epoch 133
Validation binary_cross_entropy = 0.402027
Epoch 134
Loss = 3.8293e-01, PNorm = 65.7560, GNorm = 2.7578, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.491551
Epoch 135
Validation binary_cross_entropy = 0.410273
Epoch 136
Validation binary_cross_entropy = 0.397548
Epoch 137
Validation binary_cross_entropy = 0.445451
Epoch 138
Validation binary_cross_entropy = 0.508258
Epoch 139
Loss = 2.0537e-01, PNorm = 65.8864, GNorm = 1.0507, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.481198
Epoch 140
Validation binary_cross_entropy = 0.479820
Epoch 141
Validation binary_cross_entropy = 0.526502
Epoch 142
Validation binary_cross_entropy = 0.574982
Epoch 143
Validation binary_cross_entropy = 0.612215
Epoch 144
Loss = 1.4681e-01, PNorm = 66.0114, GNorm = 2.1856, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.639077
Epoch 145
Validation binary_cross_entropy = 0.698407
Epoch 146
Validation binary_cross_entropy = 0.632611
Epoch 147
Validation binary_cross_entropy = 0.594927
Epoch 148
Validation binary_cross_entropy = 0.561614
Epoch 149
Loss = 2.0230e-01, PNorm = 66.1266, GNorm = 2.8641, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.585597
Epoch 150
Validation binary_cross_entropy = 0.540583
Epoch 151
Validation binary_cross_entropy = 0.558106
Epoch 152
Validation binary_cross_entropy = 0.586040
Epoch 153
Validation binary_cross_entropy = 0.586533
Epoch 154
Loss = 1.6963e-01, PNorm = 66.2396, GNorm = 1.3229, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.604818
Epoch 155
Validation binary_cross_entropy = 0.590216
Epoch 156
Validation binary_cross_entropy = 0.615600
Epoch 157
Validation binary_cross_entropy = 0.632788
Epoch 158
Validation binary_cross_entropy = 0.650601
Epoch 159
Loss = 2.1232e-01, PNorm = 66.3510, GNorm = 1.5713, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.663122
Epoch 160
Validation binary_cross_entropy = 0.651727
Epoch 161
Validation binary_cross_entropy = 0.442205
Epoch 162
Validation binary_cross_entropy = 0.394072
Epoch 163
Validation binary_cross_entropy = 0.399031
Epoch 164
Loss = 2.4766e-01, PNorm = 66.4620, GNorm = 1.4658, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.440432
Epoch 165
Validation binary_cross_entropy = 0.502177
Epoch 166
Validation binary_cross_entropy = 0.531155
Epoch 167
Validation binary_cross_entropy = 0.539069
Epoch 168
Validation binary_cross_entropy = 0.534165
Epoch 169
Loss = 1.5560e-01, PNorm = 66.5774, GNorm = 1.6666, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.524106
Epoch 170
Validation binary_cross_entropy = 0.523989
Epoch 171
Validation binary_cross_entropy = 0.541298
Epoch 172
Validation binary_cross_entropy = 0.577537
Epoch 173
Validation binary_cross_entropy = 0.620675
Epoch 174
Loss = 1.9123e-01, PNorm = 66.6709, GNorm = 1.7372, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 0.647582
Epoch 175
Validation binary_cross_entropy = 0.675860
Epoch 176
Validation binary_cross_entropy = 0.667419
Epoch 177
Validation binary_cross_entropy = 0.670002
Epoch 178
Validation binary_cross_entropy = 0.691040
Epoch 179
Loss = 1.5681e-01, PNorm = 66.7594, GNorm = 6.3627, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 0.679647
Epoch 180
Validation binary_cross_entropy = 0.678242
Epoch 181
Validation binary_cross_entropy = 0.640047
Epoch 182
Validation binary_cross_entropy = 0.621404
Epoch 183
Validation binary_cross_entropy = 0.580613
Epoch 184
Loss = 9.1329e-02, PNorm = 66.8666, GNorm = 2.6225, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 0.620638
Epoch 185
Validation binary_cross_entropy = 0.618826
Epoch 186
Validation binary_cross_entropy = 0.665282
Epoch 187
Validation binary_cross_entropy = 0.700099
Epoch 188
Validation binary_cross_entropy = 0.702531
Epoch 189
Loss = 6.4512e-02, PNorm = 66.9703, GNorm = 1.2093, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 0.732174
Epoch 190
Validation binary_cross_entropy = 0.732423
Epoch 191
Validation binary_cross_entropy = 0.754431
Epoch 192
Validation binary_cross_entropy = 0.707307
Epoch 193
Validation binary_cross_entropy = 0.689016
Epoch 194
Loss = 1.0651e-01, PNorm = 67.0517, GNorm = 1.0455, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 0.663620
Epoch 195
Validation binary_cross_entropy = 0.643563
Epoch 196
Validation binary_cross_entropy = 0.656336
Epoch 197
Validation binary_cross_entropy = 0.659149
Epoch 198
Validation binary_cross_entropy = 0.628559
Epoch 199
Loss = 7.2480e-02, PNorm = 67.1227, GNorm = 0.9048, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 0.639900
Epoch 200
Validation binary_cross_entropy = 0.670813
Epoch 201
Validation binary_cross_entropy = 0.733165
Epoch 202
Validation binary_cross_entropy = 0.720059
Epoch 203
Validation binary_cross_entropy = 0.740153
Epoch 204
Loss = 3.7456e-02, PNorm = 67.2031, GNorm = 0.2152, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 0.755571
Epoch 205
Validation binary_cross_entropy = 0.781123
Epoch 206
Validation binary_cross_entropy = 0.825172
Epoch 207
Validation binary_cross_entropy = 0.877504
Epoch 208
Validation binary_cross_entropy = 0.922026
Epoch 209
Loss = 1.3644e-01, PNorm = 67.2714, GNorm = 1.5548, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 0.934244
Epoch 210
Validation binary_cross_entropy = 0.936847
Epoch 211
Validation binary_cross_entropy = 0.930200
Epoch 212
Validation binary_cross_entropy = 0.921687
Epoch 213
Validation binary_cross_entropy = 0.903498
Epoch 214
Loss = 1.7617e-01, PNorm = 67.3364, GNorm = 3.6271, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 0.873738
Epoch 215
Validation binary_cross_entropy = 0.878439
Epoch 216
Validation binary_cross_entropy = 0.836911
Epoch 217
Validation binary_cross_entropy = 0.818574
Epoch 218
Validation binary_cross_entropy = 0.836896
Epoch 219
Loss = 8.2579e-02, PNorm = 67.4039, GNorm = 1.1818, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 0.870595
Epoch 220
Validation binary_cross_entropy = 0.914395
Epoch 221
Validation binary_cross_entropy = 0.913149
Epoch 222
Validation binary_cross_entropy = 0.913756
Epoch 223
Validation binary_cross_entropy = 0.924241
Epoch 224
Loss = 6.3227e-02, PNorm = 67.4713, GNorm = 3.7481, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 0.960573
Epoch 225
Validation binary_cross_entropy = 0.996536
Epoch 226
Validation binary_cross_entropy = 1.012333
Epoch 227
Validation binary_cross_entropy = 1.016157
Epoch 228
Validation binary_cross_entropy = 1.003622
Epoch 229
Loss = 1.0721e-01, PNorm = 67.5428, GNorm = 0.8506, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 1.035002
Epoch 230
Validation binary_cross_entropy = 0.956165
Epoch 231
Validation binary_cross_entropy = 0.962658
Epoch 232
Validation binary_cross_entropy = 0.968113
Epoch 233
Validation binary_cross_entropy = 0.938806
Epoch 234
Loss = 1.6827e-01, PNorm = 67.6229, GNorm = 2.1261, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 0.975307
Epoch 235
Validation binary_cross_entropy = 0.805311
Epoch 236
Validation binary_cross_entropy = 0.753996
Epoch 237
Validation binary_cross_entropy = 0.724742
Epoch 238
Validation binary_cross_entropy = 0.699178
Epoch 239
Loss = 1.7107e-01, PNorm = 67.7074, GNorm = 2.1629, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 0.739800
Epoch 240
Validation binary_cross_entropy = 0.789324
Epoch 241
Validation binary_cross_entropy = 0.750819
Epoch 242
Validation binary_cross_entropy = 0.738809
Epoch 243
Validation binary_cross_entropy = 0.745816
Epoch 244
Loss = 1.6517e-01, PNorm = 67.8062, GNorm = 1.3805, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 0.747619
Epoch 245
Validation binary_cross_entropy = 0.760351
Epoch 246
Validation binary_cross_entropy = 0.795803
Epoch 247
Validation binary_cross_entropy = 0.843399
Epoch 248
Validation binary_cross_entropy = 0.902029
Epoch 249
Loss = 1.5845e-01, PNorm = 67.8743, GNorm = 5.7840, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 0.885994
Epoch 250
Validation binary_cross_entropy = 0.837175
Epoch 251
Validation binary_cross_entropy = 0.834124
Epoch 252
Validation binary_cross_entropy = 0.863985
Epoch 253
Validation binary_cross_entropy = 0.918563
Epoch 254
Loss = 1.6137e-01, PNorm = 67.9331, GNorm = 1.0385, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 0.823608
Epoch 255
Validation binary_cross_entropy = 0.778850
Epoch 256
Validation binary_cross_entropy = 0.781743
Epoch 257
Validation binary_cross_entropy = 0.801988
Epoch 258
Validation binary_cross_entropy = 0.872942
Epoch 259
Loss = 1.3978e-01, PNorm = 67.9915, GNorm = 2.4525, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 0.981546
Epoch 260
Validation binary_cross_entropy = 1.006060
Epoch 261
Validation binary_cross_entropy = 1.036832
Epoch 262
Validation binary_cross_entropy = 1.081043
Epoch 263
Validation binary_cross_entropy = 1.143172
Epoch 264
Loss = 5.1914e-02, PNorm = 68.0605, GNorm = 1.4255, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 1.174832
Epoch 265
Validation binary_cross_entropy = 1.150185
Epoch 266
Validation binary_cross_entropy = 1.142175
Epoch 267
Validation binary_cross_entropy = 1.163554
Epoch 268
Validation binary_cross_entropy = 1.149454
Epoch 269
Loss = 3.8407e-02, PNorm = 68.1108, GNorm = 0.9273, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 1.114847
Epoch 270
Validation binary_cross_entropy = 1.091699
Epoch 271
Validation binary_cross_entropy = 1.082084
Epoch 272
Validation binary_cross_entropy = 1.086115
Epoch 273
Validation binary_cross_entropy = 1.058876
Epoch 274
Loss = 4.1368e-02, PNorm = 68.1609, GNorm = 0.8233, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 1.024378
Epoch 275
Validation binary_cross_entropy = 1.022900
Epoch 276
Validation binary_cross_entropy = 1.059739
Epoch 277
Validation binary_cross_entropy = 1.141862
Epoch 278
Validation binary_cross_entropy = 1.157985
Epoch 279
Loss = 4.0875e-02, PNorm = 68.2106, GNorm = 0.2399, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 1.032394
Epoch 280
Validation binary_cross_entropy = 1.007251
Epoch 281
Validation binary_cross_entropy = 1.044358
Epoch 282
Validation binary_cross_entropy = 1.103137
Epoch 283
Validation binary_cross_entropy = 1.031992
Epoch 284
Loss = 9.1577e-02, PNorm = 68.2593, GNorm = 0.5152, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 0.996821
Epoch 285
Validation binary_cross_entropy = 1.025843
Epoch 286
Validation binary_cross_entropy = 1.102168
Epoch 287
Validation binary_cross_entropy = 1.088647
Epoch 288
Validation binary_cross_entropy = 1.096987
Epoch 289
Loss = 6.6537e-02, PNorm = 68.3209, GNorm = 1.1367, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 1.127356
Epoch 290
Validation binary_cross_entropy = 1.191445
Epoch 291
Validation binary_cross_entropy = 1.210304
Epoch 292
Validation binary_cross_entropy = 1.259330
Epoch 293
Validation binary_cross_entropy = 1.323222
Epoch 294
Loss = 5.5544e-02, PNorm = 68.3817, GNorm = 1.4752, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 1.327587
Epoch 295
Validation binary_cross_entropy = 1.307706
Epoch 296
Validation binary_cross_entropy = 1.311044
Epoch 297
Validation binary_cross_entropy = 1.358216
Epoch 298
Validation binary_cross_entropy = 1.458270
Epoch 299
Loss = 9.2846e-02, PNorm = 68.4349, GNorm = 1.0045, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 1.349580
Model 0 best validation binary_cross_entropy = 0.288288 on epoch 7
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.244754
Ensemble test binary_cross_entropy = 0.244754
Fold 4
Splitting data with seed 4
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.4, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=800, bias=False)
        (W_h): Linear(in_features=800, out_features=800, bias=False)
        (W_o): Linear(in_features=933, out_features=800, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.4, inplace=False)
    (1): Linear(in_features=3100, out_features=800, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.4, inplace=False)
    (4): Linear(in_features=800, out_features=800, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.4, inplace=False)
    (7): Linear(in_features=800, out_features=1, bias=True)
  )
)
Number of parameters = 4,627,201
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 0.288492
Epoch 1
Validation binary_cross_entropy = 0.299842
Epoch 2
Validation binary_cross_entropy = 0.543299
Epoch 3
Validation binary_cross_entropy = 0.315291
Epoch 4
Loss = 8.1265e-01, PNorm = 63.1515, GNorm = 7.8907, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 0.296001
Epoch 5
Validation binary_cross_entropy = 0.422690
Epoch 6
Validation binary_cross_entropy = 0.719525
Epoch 7
Validation binary_cross_entropy = 0.306277
Epoch 8
Validation binary_cross_entropy = 0.396395
Epoch 9
Loss = 6.9810e-01, PNorm = 63.2896, GNorm = 6.6396, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 0.909441
Epoch 10
Validation binary_cross_entropy = 0.319208
Epoch 11
Validation binary_cross_entropy = 0.344195
Epoch 12
Validation binary_cross_entropy = 0.870440
Epoch 13
Validation binary_cross_entropy = 0.701116
Epoch 14
Loss = 8.1211e-01, PNorm = 63.4412, GNorm = 4.5585, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 0.306238
Epoch 15
Validation binary_cross_entropy = 0.491329
Epoch 16
Validation binary_cross_entropy = 0.811347
Epoch 17
Validation binary_cross_entropy = 0.366089
Epoch 18
Validation binary_cross_entropy = 0.339635
Epoch 19
Loss = 7.1207e-01, PNorm = 63.5675, GNorm = 3.5811, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 0.571972
Epoch 20
Validation binary_cross_entropy = 0.852517
Epoch 21
Validation binary_cross_entropy = 0.555954
Epoch 22
Validation binary_cross_entropy = 0.412846
Epoch 23
Validation binary_cross_entropy = 0.692608
Epoch 24
Loss = 6.7237e-01, PNorm = 63.6636, GNorm = 3.9711, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 0.614873
Epoch 25
Validation binary_cross_entropy = 0.441450
Epoch 26
Validation binary_cross_entropy = 0.397073
Epoch 27
Validation binary_cross_entropy = 0.483526
Epoch 28
Validation binary_cross_entropy = 0.552853
Epoch 29
Loss = 5.9715e-01, PNorm = 63.7372, GNorm = 2.5620, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 0.477576
Epoch 30
Validation binary_cross_entropy = 0.450970
Epoch 31
Validation binary_cross_entropy = 0.475460
Epoch 32
Validation binary_cross_entropy = 0.556908
Epoch 33
Validation binary_cross_entropy = 0.406190
Epoch 34
Loss = 6.0678e-01, PNorm = 63.8067, GNorm = 2.5643, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 0.381212
Epoch 35
Validation binary_cross_entropy = 0.412417
Epoch 36
Validation binary_cross_entropy = 0.497742
Epoch 37
Validation binary_cross_entropy = 0.578227
Epoch 38
Validation binary_cross_entropy = 0.503798
Epoch 39
Loss = 7.0062e-01, PNorm = 63.8593, GNorm = 3.6762, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 0.410573
Epoch 40
Validation binary_cross_entropy = 0.462313
Epoch 41
Validation binary_cross_entropy = 0.612047
Epoch 42
Validation binary_cross_entropy = 0.558900
Epoch 43
Validation binary_cross_entropy = 0.401490
Epoch 44
Loss = 5.9761e-01, PNorm = 63.9216, GNorm = 2.6747, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 0.397668
Epoch 45
Validation binary_cross_entropy = 0.490424
Epoch 46
Validation binary_cross_entropy = 0.620691
Epoch 47
Validation binary_cross_entropy = 0.606092
Epoch 48
Validation binary_cross_entropy = 0.553369
Epoch 49
Loss = 5.9209e-01, PNorm = 63.9908, GNorm = 1.9154, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 0.463137
Epoch 50
Validation binary_cross_entropy = 0.449482
Epoch 51
Validation binary_cross_entropy = 0.484529
Epoch 52
Validation binary_cross_entropy = 0.498411
Epoch 53
Validation binary_cross_entropy = 0.439562
Epoch 54
Loss = 5.6853e-01, PNorm = 64.0589, GNorm = 1.9765, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 0.428338
Epoch 55
Validation binary_cross_entropy = 0.488247
Epoch 56
Validation binary_cross_entropy = 0.557612
Epoch 57
Validation binary_cross_entropy = 0.533687
Epoch 58
Validation binary_cross_entropy = 0.515251
Epoch 59
Loss = 6.5696e-01, PNorm = 64.1299, GNorm = 2.6334, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 0.419050
Epoch 60
Validation binary_cross_entropy = 0.417131
Epoch 61
Validation binary_cross_entropy = 0.467819
Epoch 62
Validation binary_cross_entropy = 0.473204
Epoch 63
Validation binary_cross_entropy = 0.488440
Epoch 64
Loss = 5.2813e-01, PNorm = 64.2105, GNorm = 1.6724, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 0.547683
Epoch 65
Validation binary_cross_entropy = 0.401022
Epoch 66
Validation binary_cross_entropy = 0.297535
Epoch 67
Validation binary_cross_entropy = 0.409309
Epoch 68
Validation binary_cross_entropy = 0.669960
Epoch 69
Loss = 5.9968e-01, PNorm = 64.2814, GNorm = 2.0931, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 0.589756
Epoch 70
Validation binary_cross_entropy = 0.451441
Epoch 71
Validation binary_cross_entropy = 0.416110
Epoch 72
Validation binary_cross_entropy = 0.482423
Epoch 73
Validation binary_cross_entropy = 0.553170
Epoch 74
Loss = 5.6295e-01, PNorm = 64.3738, GNorm = 1.3523, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 0.431929
Epoch 75
Validation binary_cross_entropy = 0.379163
Epoch 76
Validation binary_cross_entropy = 0.405960
Epoch 77
Validation binary_cross_entropy = 0.412164
Epoch 78
Validation binary_cross_entropy = 0.447813
Epoch 79
Loss = 5.2126e-01, PNorm = 64.4663, GNorm = 2.0992, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 0.393825
Epoch 80
Validation binary_cross_entropy = 0.369460
Epoch 81
Validation binary_cross_entropy = 0.486076
Epoch 82
Validation binary_cross_entropy = 0.467802
Epoch 83
Validation binary_cross_entropy = 0.370557
Epoch 84
Loss = 4.3767e-01, PNorm = 64.5614, GNorm = 1.5679, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 0.472103
Epoch 85
Validation binary_cross_entropy = 0.459200
Epoch 86
Validation binary_cross_entropy = 0.339749
Epoch 87
Validation binary_cross_entropy = 0.542451
Epoch 88
Validation binary_cross_entropy = 0.441087
Epoch 89
Loss = 4.1964e-01, PNorm = 64.6586, GNorm = 2.0495, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 0.319750
Epoch 90
Validation binary_cross_entropy = 0.373408
Epoch 91
Validation binary_cross_entropy = 0.502973
Epoch 92
Validation binary_cross_entropy = 0.405448
Epoch 93
Validation binary_cross_entropy = 0.317231
Epoch 94
Loss = 4.8523e-01, PNorm = 64.7657, GNorm = 1.4608, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 0.420130
Epoch 95
Validation binary_cross_entropy = 0.429675
Epoch 96
Validation binary_cross_entropy = 0.319749
Epoch 97
Validation binary_cross_entropy = 0.380187
Epoch 98
Validation binary_cross_entropy = 0.307637
Epoch 99
Loss = 3.4253e-01, PNorm = 64.8643, GNorm = 2.1386, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.466270
Epoch 100
Validation binary_cross_entropy = 0.345845
Epoch 101
Validation binary_cross_entropy = 0.288632
Epoch 102
Validation binary_cross_entropy = 0.473014
Epoch 103
Validation binary_cross_entropy = 0.397956
Epoch 104
Loss = 3.2407e-01, PNorm = 64.9747, GNorm = 1.5386, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.344894
Epoch 105
Validation binary_cross_entropy = 0.539123
Epoch 106
Validation binary_cross_entropy = 0.433450
Epoch 107
Validation binary_cross_entropy = 0.368778
Epoch 108
Validation binary_cross_entropy = 0.463620
Epoch 109
Loss = 3.0781e-01, PNorm = 65.1021, GNorm = 2.7088, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.372240
Epoch 110
Validation binary_cross_entropy = 0.395180
Epoch 111
Validation binary_cross_entropy = 0.418839
Epoch 112
Validation binary_cross_entropy = 0.319872
Epoch 113
Validation binary_cross_entropy = 0.435981
Epoch 114
Loss = 3.5858e-01, PNorm = 65.2264, GNorm = 3.6772, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.358717
Epoch 115
Validation binary_cross_entropy = 0.303516
Epoch 116
Validation binary_cross_entropy = 0.335892
Epoch 117
Validation binary_cross_entropy = 0.413988
Epoch 118
Validation binary_cross_entropy = 0.425450
Epoch 119
Loss = 4.0860e-01, PNorm = 65.3553, GNorm = 2.6665, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.348972
Epoch 120
Validation binary_cross_entropy = 0.601076
Epoch 121
Validation binary_cross_entropy = 0.320109
Epoch 122
Validation binary_cross_entropy = 0.287189
Epoch 123
Validation binary_cross_entropy = 0.365995
Epoch 124
Loss = 3.1365e-01, PNorm = 65.4845, GNorm = 1.5758, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.553133
Epoch 125
Validation binary_cross_entropy = 0.504318
Epoch 126
Validation binary_cross_entropy = 0.347122
Epoch 127
Validation binary_cross_entropy = 0.374508
Epoch 128
Validation binary_cross_entropy = 0.486244
Epoch 129
Loss = 2.6749e-01, PNorm = 65.6170, GNorm = 2.7061, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.451229
Epoch 130
Validation binary_cross_entropy = 0.361152
Epoch 131
Validation binary_cross_entropy = 0.363787
Epoch 132
Validation binary_cross_entropy = 0.408224
Epoch 133
Validation binary_cross_entropy = 0.421603
Epoch 134
Loss = 2.5335e-01, PNorm = 65.7448, GNorm = 1.2757, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.411673
Epoch 135
Validation binary_cross_entropy = 0.432730
Epoch 136
Validation binary_cross_entropy = 0.468344
Epoch 137
Validation binary_cross_entropy = 0.480847
Epoch 138
Validation binary_cross_entropy = 0.549372
Epoch 139
Loss = 3.0719e-01, PNorm = 65.8600, GNorm = 3.6005, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.472428
Epoch 140
Validation binary_cross_entropy = 0.436960
Epoch 141
Validation binary_cross_entropy = 0.422572
Epoch 142
Validation binary_cross_entropy = 0.559141
Epoch 143
Validation binary_cross_entropy = 0.476101
Epoch 144
Loss = 1.5877e-01, PNorm = 65.9867, GNorm = 0.8794, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.457860
Epoch 145
Validation binary_cross_entropy = 0.493195
Epoch 146
Validation binary_cross_entropy = 0.514537
Epoch 147
Validation binary_cross_entropy = 0.564307
Epoch 148
Validation binary_cross_entropy = 0.577007
Epoch 149
Loss = 1.8379e-01, PNorm = 66.1100, GNorm = 4.5137, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.597148
Epoch 150
Validation binary_cross_entropy = 0.613608
Epoch 151
Validation binary_cross_entropy = 0.639884
Epoch 152
Validation binary_cross_entropy = 0.633445
Epoch 153
Validation binary_cross_entropy = 0.632491
Epoch 154
Loss = 1.1833e-01, PNorm = 66.2233, GNorm = 1.8283, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.649407
Epoch 155
Validation binary_cross_entropy = 0.676634
Epoch 156
Validation binary_cross_entropy = 0.710803
Epoch 157
Validation binary_cross_entropy = 0.708321
Epoch 158
Validation binary_cross_entropy = 0.680733
Epoch 159
Loss = 1.4231e-01, PNorm = 66.3165, GNorm = 4.4290, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.670964
Epoch 160
Validation binary_cross_entropy = 0.645692
Epoch 161
Validation binary_cross_entropy = 0.615526
Epoch 162
Validation binary_cross_entropy = 0.563780
Epoch 163
Validation binary_cross_entropy = 0.553024
Epoch 164
Loss = 2.4194e-01, PNorm = 66.4023, GNorm = 1.5600, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.494587
Epoch 165
Validation binary_cross_entropy = 0.501754
Epoch 166
Validation binary_cross_entropy = 0.497926
Epoch 167
Validation binary_cross_entropy = 0.507897
Epoch 168
Validation binary_cross_entropy = 0.542008
Epoch 169
Loss = 2.2428e-01, PNorm = 66.5016, GNorm = 1.3830, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.571778
Epoch 170
Validation binary_cross_entropy = 0.660411
Epoch 171
Validation binary_cross_entropy = 0.584957
Epoch 172
Validation binary_cross_entropy = 0.597132
Epoch 173
Validation binary_cross_entropy = 0.550206
Epoch 174
Loss = 1.2431e-01, PNorm = 66.5879, GNorm = 0.7695, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 0.601356
Epoch 175
Validation binary_cross_entropy = 0.627331
Epoch 176
Validation binary_cross_entropy = 0.572003
Epoch 177
Validation binary_cross_entropy = 0.595815
Epoch 178
Validation binary_cross_entropy = 0.602376
Epoch 179
Loss = 1.7807e-01, PNorm = 66.6700, GNorm = 1.1253, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 0.616359
Epoch 180
Validation binary_cross_entropy = 0.691526
Epoch 181
Validation binary_cross_entropy = 0.667546
Epoch 182
Validation binary_cross_entropy = 0.670233
Epoch 183
Validation binary_cross_entropy = 0.708248
Epoch 184
Loss = 1.2918e-01, PNorm = 66.7524, GNorm = 1.1723, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 0.763388
Epoch 185
Validation binary_cross_entropy = 0.787253
Epoch 186
Validation binary_cross_entropy = 0.796164
Epoch 187
Validation binary_cross_entropy = 0.846020
Epoch 188
Validation binary_cross_entropy = 0.888438
Epoch 189
Loss = 8.1052e-02, PNorm = 66.8272, GNorm = 1.2412, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 0.893048
Epoch 190
Validation binary_cross_entropy = 0.866334
Epoch 191
Validation binary_cross_entropy = 0.881472
Epoch 192
Validation binary_cross_entropy = 0.844109
Epoch 193
Validation binary_cross_entropy = 0.823337
Epoch 194
Loss = 1.7810e-01, PNorm = 66.8872, GNorm = 1.7229, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 0.738904
Epoch 195
Validation binary_cross_entropy = 0.698282
Epoch 196
Validation binary_cross_entropy = 0.697426
Epoch 197
Validation binary_cross_entropy = 0.688727
Epoch 198
Validation binary_cross_entropy = 0.647209
Epoch 199
Loss = 1.2401e-01, PNorm = 66.9585, GNorm = 1.4344, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 0.673292
Epoch 200
Validation binary_cross_entropy = 0.736361
Epoch 201
Validation binary_cross_entropy = 0.833612
Epoch 202
Validation binary_cross_entropy = 0.830946
Epoch 203
Validation binary_cross_entropy = 0.786475
Epoch 204
Loss = 6.5641e-02, PNorm = 67.0398, GNorm = 3.3544, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 0.812134
Epoch 205
Validation binary_cross_entropy = 0.881436
Epoch 206
Validation binary_cross_entropy = 0.866491
Epoch 207
Validation binary_cross_entropy = 0.842056
Epoch 208
Validation binary_cross_entropy = 0.800843
Epoch 209
Loss = 7.4235e-02, PNorm = 67.1174, GNorm = 1.1437, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 0.774655
Epoch 210
Validation binary_cross_entropy = 0.765011
Epoch 211
Validation binary_cross_entropy = 0.772774
Epoch 212
Validation binary_cross_entropy = 0.788674
Epoch 213
Validation binary_cross_entropy = 0.818022
Epoch 214
Loss = 1.6156e-01, PNorm = 67.1897, GNorm = 1.0422, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 0.860095
Epoch 215
Validation binary_cross_entropy = 0.887749
Epoch 216
Validation binary_cross_entropy = 0.879551
Epoch 217
Validation binary_cross_entropy = 0.952249
Epoch 218
Validation binary_cross_entropy = 0.940468
Epoch 219
Loss = 9.2942e-02, PNorm = 67.2484, GNorm = 2.2695, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 0.939447
Epoch 220
Validation binary_cross_entropy = 0.929503
Epoch 221
Validation binary_cross_entropy = 0.941944
Epoch 222
Validation binary_cross_entropy = 0.956086
Epoch 223
Validation binary_cross_entropy = 0.788120
Epoch 224
Loss = 5.3579e-02, PNorm = 67.3229, GNorm = 1.1557, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 0.711368
Epoch 225
Validation binary_cross_entropy = 0.696573
Epoch 226
Validation binary_cross_entropy = 0.707411
Epoch 227
Validation binary_cross_entropy = 0.650048
Epoch 228
Validation binary_cross_entropy = 0.618063
Epoch 229
Loss = 1.3217e-01, PNorm = 67.4078, GNorm = 1.2706, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 0.646415
Epoch 230
Validation binary_cross_entropy = 0.704910
Epoch 231
Validation binary_cross_entropy = 0.695859
Epoch 232
Validation binary_cross_entropy = 0.696799
Epoch 233
Validation binary_cross_entropy = 0.776171
Epoch 234
Loss = 5.0509e-02, PNorm = 67.4974, GNorm = 0.9569, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 0.886606
Epoch 235
Validation binary_cross_entropy = 0.952678
Epoch 236
Validation binary_cross_entropy = 0.862890
Epoch 237
Validation binary_cross_entropy = 0.851269
Epoch 238
Validation binary_cross_entropy = 0.834280
Epoch 239
Loss = 4.9108e-02, PNorm = 67.5672, GNorm = 0.2768, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 0.807954
Epoch 240
Validation binary_cross_entropy = 0.786730
Epoch 241
Validation binary_cross_entropy = 0.795628
Epoch 242
Validation binary_cross_entropy = 0.818009
Epoch 243
Validation binary_cross_entropy = 0.827724
Epoch 244
Loss = 5.5007e-02, PNorm = 67.6227, GNorm = 0.4361, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 0.828937
Epoch 245
Validation binary_cross_entropy = 0.841685
Epoch 246
Validation binary_cross_entropy = 0.915851
Epoch 247
Validation binary_cross_entropy = 0.939810
Epoch 248
Validation binary_cross_entropy = 0.875095
Epoch 249
Loss = 4.5072e-02, PNorm = 67.6748, GNorm = 2.0576, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 0.878501
Epoch 250
Validation binary_cross_entropy = 0.980757
Epoch 251
Validation binary_cross_entropy = 1.048447
Epoch 252
Validation binary_cross_entropy = 0.983834
Epoch 253
Validation binary_cross_entropy = 0.985679
Epoch 254
Loss = 4.4396e-02, PNorm = 67.7323, GNorm = 0.2994, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 1.050910
Epoch 255
Validation binary_cross_entropy = 1.091812
Epoch 256
Validation binary_cross_entropy = 1.088791
Epoch 257
Validation binary_cross_entropy = 1.028713
Epoch 258
Validation binary_cross_entropy = 1.025065
Epoch 259
Loss = 3.3006e-02, PNorm = 67.7895, GNorm = 0.7796, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 1.063460
Epoch 260
Validation binary_cross_entropy = 1.131845
Epoch 261
Validation binary_cross_entropy = 1.161702
Epoch 262
Validation binary_cross_entropy = 1.119502
Epoch 263
Validation binary_cross_entropy = 1.064125
Epoch 264
Loss = 1.2071e-01, PNorm = 67.8354, GNorm = 0.3764, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 0.972817
Epoch 265
Validation binary_cross_entropy = 0.998623
Epoch 266
Validation binary_cross_entropy = 1.004773
Epoch 267
Validation binary_cross_entropy = 0.964099
Epoch 268
Validation binary_cross_entropy = 0.970218
Epoch 269
Loss = 6.6356e-02, PNorm = 67.8870, GNorm = 2.9820, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 1.001966
Epoch 270
Validation binary_cross_entropy = 0.949548
Epoch 271
Validation binary_cross_entropy = 0.882026
Epoch 272
Validation binary_cross_entropy = 0.850530
Epoch 273
Validation binary_cross_entropy = 0.849377
Epoch 274
Loss = 2.1530e-02, PNorm = 67.9460, GNorm = 0.7333, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 0.859311
Epoch 275
Validation binary_cross_entropy = 0.897594
Epoch 276
Validation binary_cross_entropy = 0.988357
Epoch 277
Validation binary_cross_entropy = 0.980737
Epoch 278
Validation binary_cross_entropy = 0.999562
Epoch 279
Loss = 1.1250e-02, PNorm = 67.9977, GNorm = 0.2630, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 1.050128
Epoch 280
Validation binary_cross_entropy = 1.132589
Epoch 281
Validation binary_cross_entropy = 1.350404
Epoch 282
Validation binary_cross_entropy = 1.076796
Epoch 283
Validation binary_cross_entropy = 0.924323
Epoch 284
Loss = 2.7630e-02, PNorm = 68.0418, GNorm = 1.7022, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 0.874866
Epoch 285
Validation binary_cross_entropy = 0.879988
Epoch 286
Validation binary_cross_entropy = 0.899492
Epoch 287
Validation binary_cross_entropy = 0.888202
Epoch 288
Validation binary_cross_entropy = 0.901789
Epoch 289
Loss = 8.6714e-02, PNorm = 68.0950, GNorm = 2.2805, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 0.947751
Epoch 290
Validation binary_cross_entropy = 0.997188
Epoch 291
Validation binary_cross_entropy = 1.072040
Epoch 292
Validation binary_cross_entropy = 1.155267
Epoch 293
Validation binary_cross_entropy = 1.246546
Epoch 294
Loss = 8.2947e-02, PNorm = 68.1399, GNorm = 0.4094, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 1.090367
Epoch 295
Validation binary_cross_entropy = 1.105184
Epoch 296
Validation binary_cross_entropy = 1.107655
Epoch 297
Validation binary_cross_entropy = 1.196965
Epoch 298
Validation binary_cross_entropy = 1.177516
Epoch 299
Loss = 1.1430e-01, PNorm = 68.1812, GNorm = 1.5346, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 1.154961
Model 0 best validation binary_cross_entropy = 0.287189 on epoch 122
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.193636
Ensemble test binary_cross_entropy = 0.193636
Fold 5
Splitting data with seed 5
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.4, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=800, bias=False)
        (W_h): Linear(in_features=800, out_features=800, bias=False)
        (W_o): Linear(in_features=933, out_features=800, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.4, inplace=False)
    (1): Linear(in_features=3100, out_features=800, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.4, inplace=False)
    (4): Linear(in_features=800, out_features=800, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.4, inplace=False)
    (7): Linear(in_features=800, out_features=1, bias=True)
  )
)
Number of parameters = 4,627,201
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 0.289534
Epoch 1
Validation binary_cross_entropy = 0.303106
Epoch 2
Validation binary_cross_entropy = 0.591807
Epoch 3
Validation binary_cross_entropy = 0.330373
Epoch 4
Loss = 7.9908e-01, PNorm = 63.1519, GNorm = 7.7105, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 0.296419
Epoch 5
Validation binary_cross_entropy = 0.581647
Epoch 6
Validation binary_cross_entropy = 0.814832
Epoch 7
Validation binary_cross_entropy = 0.313778
Epoch 8
Validation binary_cross_entropy = 0.428733
Epoch 9
Loss = 7.0348e-01, PNorm = 63.2918, GNorm = 6.8510, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 0.757956
Epoch 10
Validation binary_cross_entropy = 0.331210
Epoch 11
Validation binary_cross_entropy = 0.318045
Epoch 12
Validation binary_cross_entropy = 0.867532
Epoch 13
Validation binary_cross_entropy = 0.479021
Epoch 14
Loss = 8.2486e-01, PNorm = 63.4349, GNorm = 7.0120, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 0.308802
Epoch 15
Validation binary_cross_entropy = 0.827013
Epoch 16
Validation binary_cross_entropy = 0.475661
Epoch 17
Validation binary_cross_entropy = 0.339257
Epoch 18
Validation binary_cross_entropy = 0.470448
Epoch 19
Loss = 6.7259e-01, PNorm = 63.5626, GNorm = 3.9253, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 0.482667
Epoch 20
Validation binary_cross_entropy = 0.373375
Epoch 21
Validation binary_cross_entropy = 0.426444
Epoch 22
Validation binary_cross_entropy = 0.367177
Epoch 23
Validation binary_cross_entropy = 0.346998
Epoch 24
Loss = 6.9311e-01, PNorm = 63.6544, GNorm = 3.2835, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 0.599024
Epoch 25
Validation binary_cross_entropy = 0.702147
Epoch 26
Validation binary_cross_entropy = 0.486602
Epoch 27
Validation binary_cross_entropy = 0.420818
Epoch 28
Validation binary_cross_entropy = 0.463863
Epoch 29
Loss = 7.3363e-01, PNorm = 63.7302, GNorm = 4.3377, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 0.629639
Epoch 30
Validation binary_cross_entropy = 0.407996
Epoch 31
Validation binary_cross_entropy = 0.377826
Epoch 32
Validation binary_cross_entropy = 0.461973
Epoch 33
Validation binary_cross_entropy = 0.694770
Epoch 34
Loss = 6.0167e-01, PNorm = 63.7929, GNorm = 3.0931, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 0.718450
Epoch 35
Validation binary_cross_entropy = 0.418568
Epoch 36
Validation binary_cross_entropy = 0.372510
Epoch 37
Validation binary_cross_entropy = 0.517316
Epoch 38
Validation binary_cross_entropy = 0.573922
Epoch 39
Loss = 6.2582e-01, PNorm = 63.8460, GNorm = 2.6273, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 0.506180
Epoch 40
Validation binary_cross_entropy = 0.449721
Epoch 41
Validation binary_cross_entropy = 0.456236
Epoch 42
Validation binary_cross_entropy = 0.528537
Epoch 43
Validation binary_cross_entropy = 0.576312
Epoch 44
Loss = 6.4338e-01, PNorm = 63.9045, GNorm = 2.7616, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 0.442549
Epoch 45
Validation binary_cross_entropy = 0.399084
Epoch 46
Validation binary_cross_entropy = 0.449642
Epoch 47
Validation binary_cross_entropy = 0.616498
Epoch 48
Validation binary_cross_entropy = 0.657342
Epoch 49
Loss = 5.8795e-01, PNorm = 63.9608, GNorm = 2.2239, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 0.526729
Epoch 50
Validation binary_cross_entropy = 0.453491
Epoch 51
Validation binary_cross_entropy = 0.410425
Epoch 52
Validation binary_cross_entropy = 0.480687
Epoch 53
Validation binary_cross_entropy = 0.599847
Epoch 54
Loss = 5.9766e-01, PNorm = 64.0206, GNorm = 2.2348, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 0.512638
Epoch 55
Validation binary_cross_entropy = 0.367258
Epoch 56
Validation binary_cross_entropy = 0.442367
Epoch 57
Validation binary_cross_entropy = 0.508343
Epoch 58
Validation binary_cross_entropy = 0.389574
Epoch 59
Loss = 4.5220e-01, PNorm = 64.0929, GNorm = 1.9649, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 0.609938
Epoch 60
Validation binary_cross_entropy = 0.388002
Epoch 61
Validation binary_cross_entropy = 0.379866
Epoch 62
Validation binary_cross_entropy = 0.372816
Epoch 63
Validation binary_cross_entropy = 0.473468
Epoch 64
Loss = 5.3463e-01, PNorm = 64.1623, GNorm = 2.0874, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 0.537794
Epoch 65
Validation binary_cross_entropy = 0.342271
Epoch 66
Validation binary_cross_entropy = 0.504715
Epoch 67
Validation binary_cross_entropy = 0.570594
Epoch 68
Validation binary_cross_entropy = 0.305778
Epoch 69
Loss = 5.3503e-01, PNorm = 64.2516, GNorm = 3.6953, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 0.524626
Epoch 70
Validation binary_cross_entropy = 0.718144
Epoch 71
Validation binary_cross_entropy = 0.403082
Epoch 72
Validation binary_cross_entropy = 0.344761
Epoch 73
Validation binary_cross_entropy = 0.483254
Epoch 74
Loss = 6.1048e-01, PNorm = 64.3392, GNorm = 2.8279, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 0.719330
Epoch 75
Validation binary_cross_entropy = 0.615702
Epoch 76
Validation binary_cross_entropy = 0.365460
Epoch 77
Validation binary_cross_entropy = 0.326033
Epoch 78
Validation binary_cross_entropy = 0.632319
Epoch 79
Loss = 5.5140e-01, PNorm = 64.4419, GNorm = 4.0891, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 0.348579
Epoch 80
Validation binary_cross_entropy = 0.317703
Epoch 81
Validation binary_cross_entropy = 0.589992
Epoch 82
Validation binary_cross_entropy = 0.494011
Epoch 83
Validation binary_cross_entropy = 0.358935
Epoch 84
Loss = 5.2401e-01, PNorm = 64.5326, GNorm = 2.0422, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 0.335201
Epoch 85
Validation binary_cross_entropy = 0.353149
Epoch 86
Validation binary_cross_entropy = 0.439620
Epoch 87
Validation binary_cross_entropy = 0.475514
Epoch 88
Validation binary_cross_entropy = 0.340624
Epoch 89
Loss = 4.3803e-01, PNorm = 64.6462, GNorm = 1.6727, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 0.349045
Epoch 90
Validation binary_cross_entropy = 0.380066
Epoch 91
Validation binary_cross_entropy = 0.366312
Epoch 92
Validation binary_cross_entropy = 0.343032
Epoch 93
Validation binary_cross_entropy = 0.418403
Epoch 94
Loss = 4.7908e-01, PNorm = 64.7488, GNorm = 2.7550, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 0.304596
Epoch 95
Validation binary_cross_entropy = 0.359888
Epoch 96
Validation binary_cross_entropy = 0.406973
Epoch 97
Validation binary_cross_entropy = 0.469343
Epoch 98
Validation binary_cross_entropy = 0.305717
Epoch 99
Loss = 4.1188e-01, PNorm = 64.8650, GNorm = 3.1658, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.373944
Epoch 100
Validation binary_cross_entropy = 0.339074
Epoch 101
Validation binary_cross_entropy = 0.419456
Epoch 102
Validation binary_cross_entropy = 0.391161
Epoch 103
Validation binary_cross_entropy = 0.444045
Epoch 104
Loss = 4.3906e-01, PNorm = 64.9807, GNorm = 2.7561, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.333910
Epoch 105
Validation binary_cross_entropy = 0.528069
Epoch 106
Validation binary_cross_entropy = 0.380991
Epoch 107
Validation binary_cross_entropy = 0.283916
Epoch 108
Validation binary_cross_entropy = 0.322393
Epoch 109
Loss = 3.7935e-01, PNorm = 65.0981, GNorm = 1.8180, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.461531
Epoch 110
Validation binary_cross_entropy = 0.351225
Epoch 111
Validation binary_cross_entropy = 0.306402
Epoch 112
Validation binary_cross_entropy = 0.385776
Epoch 113
Validation binary_cross_entropy = 0.452676
Epoch 114
Loss = 3.3111e-01, PNorm = 65.2166, GNorm = 1.5309, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.330124
Epoch 115
Validation binary_cross_entropy = 0.363563
Epoch 116
Validation binary_cross_entropy = 0.395418
Epoch 117
Validation binary_cross_entropy = 0.396988
Epoch 118
Validation binary_cross_entropy = 0.398406
Epoch 119
Loss = 2.2646e-01, PNorm = 65.3474, GNorm = 1.9135, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.725368
Epoch 120
Validation binary_cross_entropy = 0.425772
Epoch 121
Validation binary_cross_entropy = 0.395078
Epoch 122
Validation binary_cross_entropy = 0.446521
Epoch 123
Validation binary_cross_entropy = 0.327633
Epoch 124
Loss = 3.6488e-01, PNorm = 65.4679, GNorm = 2.2302, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.322538
Epoch 125
Validation binary_cross_entropy = 0.393764
Epoch 126
Validation binary_cross_entropy = 0.426126
Epoch 127
Validation binary_cross_entropy = 0.393221
Epoch 128
Validation binary_cross_entropy = 0.438141
Epoch 129
Loss = 2.6660e-01, PNorm = 65.6184, GNorm = 2.3785, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.558798
Epoch 130
Validation binary_cross_entropy = 0.497348
Epoch 131
Validation binary_cross_entropy = 0.737775
Epoch 132
Validation binary_cross_entropy = 0.519649
Epoch 133
Validation binary_cross_entropy = 0.451083
Epoch 134
Loss = 2.3550e-01, PNorm = 65.7456, GNorm = 1.1615, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.483131
Epoch 135
Validation binary_cross_entropy = 0.556122
Epoch 136
Validation binary_cross_entropy = 0.431436
Epoch 137
Validation binary_cross_entropy = 0.420294
Epoch 138
Validation binary_cross_entropy = 0.467153
Epoch 139
Loss = 1.8987e-01, PNorm = 65.8634, GNorm = 1.2742, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.565086
Epoch 140
Validation binary_cross_entropy = 0.539815
Epoch 141
Validation binary_cross_entropy = 0.558009
Epoch 142
Validation binary_cross_entropy = 0.595306
Epoch 143
Validation binary_cross_entropy = 0.583532
Epoch 144
Loss = 2.5914e-01, PNorm = 65.9621, GNorm = 1.3477, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.554689
Epoch 145
Validation binary_cross_entropy = 0.535855
Epoch 146
Validation binary_cross_entropy = 0.530141
Epoch 147
Validation binary_cross_entropy = 0.516437
Epoch 148
Validation binary_cross_entropy = 0.532888
Epoch 149
Loss = 2.7930e-01, PNorm = 66.0599, GNorm = 2.5731, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.547819
Epoch 150
Validation binary_cross_entropy = 0.623586
Epoch 151
Validation binary_cross_entropy = 0.617215
Epoch 152
Validation binary_cross_entropy = 0.631802
Epoch 153
Validation binary_cross_entropy = 0.614046
Epoch 154
Loss = 1.8617e-01, PNorm = 66.1565, GNorm = 1.0143, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.614184
Epoch 155
Validation binary_cross_entropy = 0.582074
Epoch 156
Validation binary_cross_entropy = 0.636387
Epoch 157
Validation binary_cross_entropy = 0.591666
Epoch 158
Validation binary_cross_entropy = 0.612196
Epoch 159
Loss = 1.4365e-01, PNorm = 66.2581, GNorm = 2.6039, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.593060
Epoch 160
Validation binary_cross_entropy = 0.617739
Epoch 161
Validation binary_cross_entropy = 0.645798
Epoch 162
Validation binary_cross_entropy = 0.712961
Epoch 163
Validation binary_cross_entropy = 0.653916
Epoch 164
Loss = 1.4867e-01, PNorm = 66.3599, GNorm = 2.2036, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.673173
Epoch 165
Validation binary_cross_entropy = 0.686101
Epoch 166
Validation binary_cross_entropy = 0.636391
Epoch 167
Validation binary_cross_entropy = 0.605493
Epoch 168
Validation binary_cross_entropy = 0.611337
Epoch 169
Loss = 1.2600e-01, PNorm = 66.4578, GNorm = 0.7406, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.634608
Epoch 170
Validation binary_cross_entropy = 0.656961
Epoch 171
Validation binary_cross_entropy = 0.705853
Epoch 172
Validation binary_cross_entropy = 0.740981
Epoch 173
Validation binary_cross_entropy = 0.811262
Epoch 174
Loss = 1.8654e-01, PNorm = 66.5507, GNorm = 0.7556, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 0.690891
Epoch 175
Validation binary_cross_entropy = 0.662704
Epoch 176
Validation binary_cross_entropy = 0.685672
Epoch 177
Validation binary_cross_entropy = 0.692484
Epoch 178
Validation binary_cross_entropy = 0.660337
Epoch 179
Loss = 1.0502e-01, PNorm = 66.6497, GNorm = 1.1772, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 0.752122
Epoch 180
Validation binary_cross_entropy = 0.722603
Epoch 181
Validation binary_cross_entropy = 0.768387
Epoch 182
Validation binary_cross_entropy = 0.741738
Epoch 183
Validation binary_cross_entropy = 0.625711
Epoch 184
Loss = 2.2870e-01, PNorm = 66.7444, GNorm = 3.7996, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 0.590199
Epoch 185
Validation binary_cross_entropy = 0.537927
Epoch 186
Validation binary_cross_entropy = 0.553995
Epoch 187
Validation binary_cross_entropy = 0.579386
Epoch 188
Validation binary_cross_entropy = 0.607008
Epoch 189
Loss = 1.0560e-01, PNorm = 66.8375, GNorm = 0.8903, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 0.666027
Epoch 190
Validation binary_cross_entropy = 0.688308
Epoch 191
Validation binary_cross_entropy = 0.703030
Epoch 192
Validation binary_cross_entropy = 0.736917
Epoch 193
Validation binary_cross_entropy = 0.739970
Epoch 194
Loss = 1.3045e-01, PNorm = 66.9184, GNorm = 2.0950, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 0.750743
Epoch 195
Validation binary_cross_entropy = 0.793034
Epoch 196
Validation binary_cross_entropy = 0.852751
Epoch 197
Validation binary_cross_entropy = 0.874709
Epoch 198
Validation binary_cross_entropy = 0.913841
Epoch 199
Loss = 9.6365e-02, PNorm = 66.9964, GNorm = 0.9199, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 0.928183
Epoch 200
Validation binary_cross_entropy = 0.964770
Epoch 201
Validation binary_cross_entropy = 0.992088
Epoch 202
Validation binary_cross_entropy = 0.906736
Epoch 203
Validation binary_cross_entropy = 0.840484
Epoch 204
Loss = 1.5256e-01, PNorm = 67.0575, GNorm = 1.4163, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 0.799154
Epoch 205
Validation binary_cross_entropy = 0.770710
Epoch 206
Validation binary_cross_entropy = 0.694112
Epoch 207
Validation binary_cross_entropy = 0.690733
Epoch 208
Validation binary_cross_entropy = 0.684760
Epoch 209
Loss = 1.5869e-01, PNorm = 67.1452, GNorm = 1.2159, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 0.689744
Epoch 210
Validation binary_cross_entropy = 0.743307
Epoch 211
Validation binary_cross_entropy = 0.740692
Epoch 212
Validation binary_cross_entropy = 0.710989
Epoch 213
Validation binary_cross_entropy = 0.729292
Epoch 214
Loss = 9.0190e-02, PNorm = 67.2293, GNorm = 1.6501, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 0.773046
Epoch 215
Validation binary_cross_entropy = 0.850831
Epoch 216
Validation binary_cross_entropy = 0.977448
Epoch 217
Validation binary_cross_entropy = 1.042622
Epoch 218
Validation binary_cross_entropy = 0.915854
Epoch 219
Loss = 1.1593e-01, PNorm = 67.3031, GNorm = 1.5969, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 0.899561
Epoch 220
Validation binary_cross_entropy = 0.933875
Epoch 221
Validation binary_cross_entropy = 1.011824
Epoch 222
Validation binary_cross_entropy = 0.887529
Epoch 223
Validation binary_cross_entropy = 0.798345
Epoch 224
Loss = 2.6421e-01, PNorm = 67.3826, GNorm = 3.1759, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 0.718095
Epoch 225
Validation binary_cross_entropy = 0.626909
Epoch 226
Validation binary_cross_entropy = 0.623843
Epoch 227
Validation binary_cross_entropy = 0.628944
Epoch 228
Validation binary_cross_entropy = 0.638889
Epoch 229
Loss = 1.1235e-01, PNorm = 67.4683, GNorm = 0.9846, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 0.666587
Epoch 230
Validation binary_cross_entropy = 0.720508
Epoch 231
Validation binary_cross_entropy = 0.802497
Epoch 232
Validation binary_cross_entropy = 0.925541
Epoch 233
Validation binary_cross_entropy = 1.054983
Epoch 234
Loss = 1.2048e-01, PNorm = 67.5443, GNorm = 2.8026, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 0.991731
Epoch 235
Validation binary_cross_entropy = 0.983692
Epoch 236
Validation binary_cross_entropy = 1.019579
Epoch 237
Validation binary_cross_entropy = 1.046572
Epoch 238
Validation binary_cross_entropy = 1.068115
Epoch 239
Loss = 1.0555e-01, PNorm = 67.6055, GNorm = 0.5131, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 1.073397
Epoch 240
Validation binary_cross_entropy = 1.095601
Epoch 241
Validation binary_cross_entropy = 1.019189
Epoch 242
Validation binary_cross_entropy = 0.952670
Epoch 243
Validation binary_cross_entropy = 0.917772
Epoch 244
Loss = 1.6130e-01, PNorm = 67.6691, GNorm = 3.0270, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 0.896089
Epoch 245
Validation binary_cross_entropy = 0.965748
Epoch 246
Validation binary_cross_entropy = 0.921935
Epoch 247
Validation binary_cross_entropy = 0.864383
Epoch 248
Validation binary_cross_entropy = 0.861341
Epoch 249
Loss = 1.2510e-01, PNorm = 67.7294, GNorm = 1.6486, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 0.909285
Epoch 250
Validation binary_cross_entropy = 0.975559
Epoch 251
Validation binary_cross_entropy = 1.029051
Epoch 252
Validation binary_cross_entropy = 1.051942
Epoch 253
Validation binary_cross_entropy = 1.028209
Epoch 254
Loss = 4.6535e-02, PNorm = 67.7896, GNorm = 0.8437, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 1.044429
Epoch 255
Validation binary_cross_entropy = 1.108052
Epoch 256
Validation binary_cross_entropy = 1.217584
Epoch 257
Validation binary_cross_entropy = 1.258634
Epoch 258
Validation binary_cross_entropy = 1.113582
Epoch 259
Loss = 1.9162e-01, PNorm = 67.8421, GNorm = 1.9296, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 0.957873
Epoch 260
Validation binary_cross_entropy = 0.875512
Epoch 261
Validation binary_cross_entropy = 0.863050
Epoch 262
Validation binary_cross_entropy = 0.868029
Epoch 263
Validation binary_cross_entropy = 0.853969
Epoch 264
Loss = 9.6258e-02, PNorm = 67.9025, GNorm = 0.8895, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 0.842791
Epoch 265
Validation binary_cross_entropy = 0.853808
Epoch 266
Validation binary_cross_entropy = 0.926803
Epoch 267
Validation binary_cross_entropy = 1.036205
Epoch 268
Validation binary_cross_entropy = 1.101811
Epoch 269
Loss = 4.7476e-02, PNorm = 67.9559, GNorm = 1.1950, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 1.068694
Epoch 270
Validation binary_cross_entropy = 1.070621
Epoch 271
Validation binary_cross_entropy = 1.080504
Epoch 272
Validation binary_cross_entropy = 1.150066
Epoch 273
Validation binary_cross_entropy = 1.288207
Epoch 274
Loss = 9.9065e-02, PNorm = 68.0014, GNorm = 5.5152, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 1.286754
Epoch 275
Validation binary_cross_entropy = 1.195713
Epoch 276
Validation binary_cross_entropy = 1.163258
Epoch 277
Validation binary_cross_entropy = 1.175258
Epoch 278
Validation binary_cross_entropy = 1.191755
Epoch 279
Loss = 8.4779e-02, PNorm = 68.0459, GNorm = 1.1401, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 1.257287
Epoch 280
Validation binary_cross_entropy = 1.257581
Epoch 281
Validation binary_cross_entropy = 1.193638
Epoch 282
Validation binary_cross_entropy = 1.171147
Epoch 283
Validation binary_cross_entropy = 1.180408
Epoch 284
Loss = 1.3435e-01, PNorm = 68.0841, GNorm = 1.9187, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 1.184433
Epoch 285
Validation binary_cross_entropy = 1.217888
Epoch 286
Validation binary_cross_entropy = 1.231984
Epoch 287
Validation binary_cross_entropy = 1.113955
Epoch 288
Validation binary_cross_entropy = 1.041147
Epoch 289
Loss = 5.8976e-02, PNorm = 68.1353, GNorm = 0.8985, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 1.037846
Epoch 290
Validation binary_cross_entropy = 1.024491
Epoch 291
Validation binary_cross_entropy = 1.065444
Epoch 292
Validation binary_cross_entropy = 1.081721
Epoch 293
Validation binary_cross_entropy = 1.018857
Epoch 294
Loss = 4.7834e-02, PNorm = 68.1844, GNorm = 0.4974, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 0.998081
Epoch 295
Validation binary_cross_entropy = 0.990471
Epoch 296
Validation binary_cross_entropy = 1.017386
Epoch 297
Validation binary_cross_entropy = 1.089213
Epoch 298
Validation binary_cross_entropy = 1.123882
Epoch 299
Loss = 4.1473e-02, PNorm = 68.2332, GNorm = 0.9392, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 1.135837
Model 0 best validation binary_cross_entropy = 0.283916 on epoch 107
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.211122
Ensemble test binary_cross_entropy = 0.211122
Fold 6
Splitting data with seed 6
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.4, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=800, bias=False)
        (W_h): Linear(in_features=800, out_features=800, bias=False)
        (W_o): Linear(in_features=933, out_features=800, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.4, inplace=False)
    (1): Linear(in_features=3100, out_features=800, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.4, inplace=False)
    (4): Linear(in_features=800, out_features=800, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.4, inplace=False)
    (7): Linear(in_features=800, out_features=1, bias=True)
  )
)
Number of parameters = 4,627,201
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 0.293489
Epoch 1
Validation binary_cross_entropy = 0.301747
Epoch 2
Validation binary_cross_entropy = 0.614025
Epoch 3
Validation binary_cross_entropy = 0.314358
Epoch 4
Loss = 9.0234e-01, PNorm = 63.1523, GNorm = 8.5275, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 0.292968
Epoch 5
Validation binary_cross_entropy = 0.569553
Epoch 6
Validation binary_cross_entropy = 0.701773
Epoch 7
Validation binary_cross_entropy = 0.312632
Epoch 8
Validation binary_cross_entropy = 0.472939
Epoch 9
Loss = 7.2079e-01, PNorm = 63.2846, GNorm = 5.3228, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 0.644445
Epoch 10
Validation binary_cross_entropy = 0.339892
Epoch 11
Validation binary_cross_entropy = 0.354199
Epoch 12
Validation binary_cross_entropy = 0.683794
Epoch 13
Validation binary_cross_entropy = 0.325539
Epoch 14
Loss = 7.7695e-01, PNorm = 63.4385, GNorm = 5.8793, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 0.317181
Epoch 15
Validation binary_cross_entropy = 1.107939
Epoch 16
Validation binary_cross_entropy = 0.365324
Epoch 17
Validation binary_cross_entropy = 0.296562
Epoch 18
Validation binary_cross_entropy = 0.521165
Epoch 19
Loss = 7.6414e-01, PNorm = 63.5639, GNorm = 6.9624, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 1.012209
Epoch 20
Validation binary_cross_entropy = 0.359591
Epoch 21
Validation binary_cross_entropy = 0.302680
Epoch 22
Validation binary_cross_entropy = 0.396236
Epoch 23
Validation binary_cross_entropy = 0.675818
Epoch 24
Loss = 6.7859e-01, PNorm = 63.6587, GNorm = 4.6212, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 0.603887
Epoch 25
Validation binary_cross_entropy = 0.344314
Epoch 26
Validation binary_cross_entropy = 0.344532
Epoch 27
Validation binary_cross_entropy = 0.449716
Epoch 28
Validation binary_cross_entropy = 0.545673
Epoch 29
Loss = 6.6723e-01, PNorm = 63.7292, GNorm = 2.7616, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 0.454104
Epoch 30
Validation binary_cross_entropy = 0.403414
Epoch 31
Validation binary_cross_entropy = 0.404294
Epoch 32
Validation binary_cross_entropy = 0.496384
Epoch 33
Validation binary_cross_entropy = 0.497117
Epoch 34
Loss = 5.5663e-01, PNorm = 63.7932, GNorm = 2.4544, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 0.546613
Epoch 35
Validation binary_cross_entropy = 0.527903
Epoch 36
Validation binary_cross_entropy = 0.482495
Epoch 37
Validation binary_cross_entropy = 0.515088
Epoch 38
Validation binary_cross_entropy = 0.480556
Epoch 39
Loss = 6.6443e-01, PNorm = 63.8496, GNorm = 3.0463, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 0.417855
Epoch 40
Validation binary_cross_entropy = 0.428011
Epoch 41
Validation binary_cross_entropy = 0.596220
Epoch 42
Validation binary_cross_entropy = 0.606077
Epoch 43
Validation binary_cross_entropy = 0.438176
Epoch 44
Loss = 5.6428e-01, PNorm = 63.9044, GNorm = 3.1183, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 0.406244
Epoch 45
Validation binary_cross_entropy = 0.545882
Epoch 46
Validation binary_cross_entropy = 0.577568
Epoch 47
Validation binary_cross_entropy = 0.458864
Epoch 48
Validation binary_cross_entropy = 0.465031
Epoch 49
Loss = 5.6304e-01, PNorm = 63.9610, GNorm = 2.1331, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 0.493329
Epoch 50
Validation binary_cross_entropy = 0.531007
Epoch 51
Validation binary_cross_entropy = 0.533820
Epoch 52
Validation binary_cross_entropy = 0.491103
Epoch 53
Validation binary_cross_entropy = 0.419268
Epoch 54
Loss = 5.3976e-01, PNorm = 64.0153, GNorm = 2.2686, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 0.440039
Epoch 55
Validation binary_cross_entropy = 0.537297
Epoch 56
Validation binary_cross_entropy = 0.549532
Epoch 57
Validation binary_cross_entropy = 0.469908
Epoch 58
Validation binary_cross_entropy = 0.411494
Epoch 59
Loss = 6.4141e-01, PNorm = 64.0731, GNorm = 2.3230, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 0.447328
Epoch 60
Validation binary_cross_entropy = 0.597291
Epoch 61
Validation binary_cross_entropy = 0.660272
Epoch 62
Validation binary_cross_entropy = 0.548013
Epoch 63
Validation binary_cross_entropy = 0.441398
Epoch 64
Loss = 6.0195e-01, PNorm = 64.1460, GNorm = 2.3538, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 0.392233
Epoch 65
Validation binary_cross_entropy = 0.438474
Epoch 66
Validation binary_cross_entropy = 0.580010
Epoch 67
Validation binary_cross_entropy = 0.697825
Epoch 68
Validation binary_cross_entropy = 0.525521
Epoch 69
Loss = 5.3425e-01, PNorm = 64.2159, GNorm = 2.1359, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 0.393212
Epoch 70
Validation binary_cross_entropy = 0.409103
Epoch 71
Validation binary_cross_entropy = 0.491681
Epoch 72
Validation binary_cross_entropy = 0.414319
Epoch 73
Validation binary_cross_entropy = 0.476861
Epoch 74
Loss = 5.8340e-01, PNorm = 64.2903, GNorm = 2.1058, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 0.682040
Epoch 75
Validation binary_cross_entropy = 0.647853
Epoch 76
Validation binary_cross_entropy = 0.420309
Epoch 77
Validation binary_cross_entropy = 0.343876
Epoch 78
Validation binary_cross_entropy = 0.610494
Epoch 79
Loss = 6.4652e-01, PNorm = 64.3904, GNorm = 3.0685, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 0.288250
Epoch 80
Validation binary_cross_entropy = 0.387944
Epoch 81
Validation binary_cross_entropy = 0.597684
Epoch 82
Validation binary_cross_entropy = 0.712290
Epoch 83
Validation binary_cross_entropy = 0.592159
Epoch 84
Loss = 5.2166e-01, PNorm = 64.4952, GNorm = 1.4487, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 0.446120
Epoch 85
Validation binary_cross_entropy = 0.394136
Epoch 86
Validation binary_cross_entropy = 0.471795
Epoch 87
Validation binary_cross_entropy = 0.539885
Epoch 88
Validation binary_cross_entropy = 0.479185
Epoch 89
Loss = 4.4481e-01, PNorm = 64.6165, GNorm = 1.6449, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 0.408270
Epoch 90
Validation binary_cross_entropy = 0.425316
Epoch 91
Validation binary_cross_entropy = 0.495722
Epoch 92
Validation binary_cross_entropy = 0.386988
Epoch 93
Validation binary_cross_entropy = 0.410314
Epoch 94
Loss = 3.9783e-01, PNorm = 64.7317, GNorm = 2.5933, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 0.363628
Epoch 95
Validation binary_cross_entropy = 0.378598
Epoch 96
Validation binary_cross_entropy = 0.472420
Epoch 97
Validation binary_cross_entropy = 0.365591
Epoch 98
Validation binary_cross_entropy = 0.430086
Epoch 99
Loss = 3.8539e-01, PNorm = 64.8403, GNorm = 1.8466, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.459187
Epoch 100
Validation binary_cross_entropy = 0.399705
Epoch 101
Validation binary_cross_entropy = 0.397056
Epoch 102
Validation binary_cross_entropy = 0.556238
Epoch 103
Validation binary_cross_entropy = 0.377706
Epoch 104
Loss = 3.2994e-01, PNorm = 64.9784, GNorm = 2.0063, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.589278
Epoch 105
Validation binary_cross_entropy = 0.305979
Epoch 106
Validation binary_cross_entropy = 0.376418
Epoch 107
Validation binary_cross_entropy = 0.469289
Epoch 108
Validation binary_cross_entropy = 0.337849
Epoch 109
Loss = 3.4071e-01, PNorm = 65.1220, GNorm = 1.5729, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.324955
Epoch 110
Validation binary_cross_entropy = 0.465295
Epoch 111
Validation binary_cross_entropy = 0.335518
Epoch 112
Validation binary_cross_entropy = 0.303083
Epoch 113
Validation binary_cross_entropy = 0.413269
Epoch 114
Loss = 3.2908e-01, PNorm = 65.2556, GNorm = 1.8398, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.335452
Epoch 115
Validation binary_cross_entropy = 0.301535
Epoch 116
Validation binary_cross_entropy = 0.520122
Epoch 117
Validation binary_cross_entropy = 0.366098
Epoch 118
Validation binary_cross_entropy = 0.312534
Epoch 119
Loss = 3.1951e-01, PNorm = 65.4008, GNorm = 2.6826, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.387874
Epoch 120
Validation binary_cross_entropy = 0.470964
Epoch 121
Validation binary_cross_entropy = 0.385001
Epoch 122
Validation binary_cross_entropy = 0.404180
Epoch 123
Validation binary_cross_entropy = 0.415754
Epoch 124
Loss = 2.8206e-01, PNorm = 65.5385, GNorm = 1.8658, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.418048
Epoch 125
Validation binary_cross_entropy = 0.387753
Epoch 126
Validation binary_cross_entropy = 0.417011
Epoch 127
Validation binary_cross_entropy = 0.365134
Epoch 128
Validation binary_cross_entropy = 0.374518
Epoch 129
Loss = 2.4043e-01, PNorm = 65.6751, GNorm = 1.8044, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.420426
Epoch 130
Validation binary_cross_entropy = 0.451899
Epoch 131
Validation binary_cross_entropy = 0.512231
Epoch 132
Validation binary_cross_entropy = 0.508906
Epoch 133
Validation binary_cross_entropy = 0.396655
Epoch 134
Loss = 2.0624e-01, PNorm = 65.7930, GNorm = 3.7620, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.368230
Epoch 135
Validation binary_cross_entropy = 0.370034
Epoch 136
Validation binary_cross_entropy = 0.375058
Epoch 137
Validation binary_cross_entropy = 0.389770
Epoch 138
Validation binary_cross_entropy = 0.430091
Epoch 139
Loss = 2.7396e-01, PNorm = 65.9172, GNorm = 3.2256, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.472394
Epoch 140
Validation binary_cross_entropy = 0.529886
Epoch 141
Validation binary_cross_entropy = 0.490586
Epoch 142
Validation binary_cross_entropy = 0.475437
Epoch 143
Validation binary_cross_entropy = 0.475941
Epoch 144
Loss = 2.0885e-01, PNorm = 66.0169, GNorm = 1.4926, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.433985
Epoch 145
Validation binary_cross_entropy = 0.439605
Epoch 146
Validation binary_cross_entropy = 0.454332
Epoch 147
Validation binary_cross_entropy = 0.497678
Epoch 148
Validation binary_cross_entropy = 0.543095
Epoch 149
Loss = 1.8164e-01, PNorm = 66.1313, GNorm = 2.3805, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.623861
Epoch 150
Validation binary_cross_entropy = 0.595238
Epoch 151
Validation binary_cross_entropy = 0.583996
Epoch 152
Validation binary_cross_entropy = 0.694042
Epoch 153
Validation binary_cross_entropy = 0.432960
Epoch 154
Loss = 3.0347e-01, PNorm = 66.2375, GNorm = 4.2439, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.425166
Epoch 155
Validation binary_cross_entropy = 0.375888
Epoch 156
Validation binary_cross_entropy = 0.420325
Epoch 157
Validation binary_cross_entropy = 0.466099
Epoch 158
Validation binary_cross_entropy = 0.536407
Epoch 159
Loss = 2.7980e-01, PNorm = 66.3818, GNorm = 1.2861, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.493207
Epoch 160
Validation binary_cross_entropy = 0.453295
Epoch 161
Validation binary_cross_entropy = 0.398143
Epoch 162
Validation binary_cross_entropy = 0.408703
Epoch 163
Validation binary_cross_entropy = 0.420514
Epoch 164
Loss = 1.6335e-01, PNorm = 66.4884, GNorm = 0.9692, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.457490
Epoch 165
Validation binary_cross_entropy = 0.526412
Epoch 166
Validation binary_cross_entropy = 0.555573
Epoch 167
Validation binary_cross_entropy = 0.598804
Epoch 168
Validation binary_cross_entropy = 0.684543
Epoch 169
Loss = 1.2683e-01, PNorm = 66.5932, GNorm = 2.1735, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.732442
Epoch 170
Validation binary_cross_entropy = 0.769169
Epoch 171
Validation binary_cross_entropy = 0.779376
Epoch 172
Validation binary_cross_entropy = 0.823284
Epoch 173
Validation binary_cross_entropy = 0.650657
Epoch 174
Loss = 1.7297e-01, PNorm = 66.6783, GNorm = 2.7252, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 0.623276
Epoch 175
Validation binary_cross_entropy = 0.513569
Epoch 176
Validation binary_cross_entropy = 0.495311
Epoch 177
Validation binary_cross_entropy = 0.538038
Epoch 178
Validation binary_cross_entropy = 0.600122
Epoch 179
Loss = 2.0669e-01, PNorm = 66.7669, GNorm = 2.9143, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 0.588863
Epoch 180
Validation binary_cross_entropy = 0.547671
Epoch 181
Validation binary_cross_entropy = 0.551308
Epoch 182
Validation binary_cross_entropy = 0.580595
Epoch 183
Validation binary_cross_entropy = 0.653159
Epoch 184
Loss = 1.1277e-01, PNorm = 66.8584, GNorm = 1.9201, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 0.670056
Epoch 185
Validation binary_cross_entropy = 0.690236
Epoch 186
Validation binary_cross_entropy = 0.730153
Epoch 187
Validation binary_cross_entropy = 0.697845
Epoch 188
Validation binary_cross_entropy = 0.709235
Epoch 189
Loss = 1.4351e-01, PNorm = 66.9443, GNorm = 1.6607, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 0.682350
Epoch 190
Validation binary_cross_entropy = 0.683389
Epoch 191
Validation binary_cross_entropy = 0.713247
Epoch 192
Validation binary_cross_entropy = 0.718493
Epoch 193
Validation binary_cross_entropy = 0.732184
Epoch 194
Loss = 9.9132e-02, PNorm = 67.0187, GNorm = 1.3884, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 0.761181
Epoch 195
Validation binary_cross_entropy = 0.778129
Epoch 196
Validation binary_cross_entropy = 0.787114
Epoch 197
Validation binary_cross_entropy = 0.803544
Epoch 198
Validation binary_cross_entropy = 0.781996
Epoch 199
Loss = 7.4685e-02, PNorm = 67.0802, GNorm = 0.5584, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 0.653189
Epoch 200
Validation binary_cross_entropy = 0.634238
Epoch 201
Validation binary_cross_entropy = 0.564254
Epoch 202
Validation binary_cross_entropy = 0.562341
Epoch 203
Validation binary_cross_entropy = 0.580927
Epoch 204
Loss = 1.9451e-01, PNorm = 67.1443, GNorm = 0.5528, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 0.579791
Epoch 205
Validation binary_cross_entropy = 0.606667
Epoch 206
Validation binary_cross_entropy = 0.614933
Epoch 207
Validation binary_cross_entropy = 0.645336
Epoch 208
Validation binary_cross_entropy = 0.714849
Epoch 209
Loss = 9.1322e-02, PNorm = 67.2126, GNorm = 1.1719, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 0.756040
Epoch 210
Validation binary_cross_entropy = 0.703131
Epoch 211
Validation binary_cross_entropy = 0.702410
Epoch 212
Validation binary_cross_entropy = 0.740334
Epoch 213
Validation binary_cross_entropy = 0.795591
Epoch 214
Loss = 1.0825e-01, PNorm = 67.2867, GNorm = 1.2293, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 0.776975
Epoch 215
Validation binary_cross_entropy = 0.746601
Epoch 216
Validation binary_cross_entropy = 0.747185
Epoch 217
Validation binary_cross_entropy = 0.761786
Epoch 218
Validation binary_cross_entropy = 0.779205
Epoch 219
Loss = 4.9489e-02, PNorm = 67.3589, GNorm = 1.6405, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 0.769287
Epoch 220
Validation binary_cross_entropy = 0.754499
Epoch 221
Validation binary_cross_entropy = 0.772852
Epoch 222
Validation binary_cross_entropy = 0.803626
Epoch 223
Validation binary_cross_entropy = 0.815199
Epoch 224
Loss = 6.0981e-02, PNorm = 67.4177, GNorm = 1.4392, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 0.793646
Epoch 225
Validation binary_cross_entropy = 0.771697
Epoch 226
Validation binary_cross_entropy = 0.769642
Epoch 227
Validation binary_cross_entropy = 0.821159
Epoch 228
Validation binary_cross_entropy = 0.826643
Epoch 229
Loss = 2.1846e-02, PNorm = 67.4676, GNorm = 0.4477, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 0.796813
Epoch 230
Validation binary_cross_entropy = 0.786348
Epoch 231
Validation binary_cross_entropy = 0.783144
Epoch 232
Validation binary_cross_entropy = 0.780293
Epoch 233
Validation binary_cross_entropy = 0.777062
Epoch 234
Loss = 1.0163e-01, PNorm = 67.5226, GNorm = 1.9619, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 0.831205
Epoch 235
Validation binary_cross_entropy = 0.811773
Epoch 236
Validation binary_cross_entropy = 0.748357
Epoch 237
Validation binary_cross_entropy = 0.740677
Epoch 238
Validation binary_cross_entropy = 0.759986
Epoch 239
Loss = 4.7129e-02, PNorm = 67.5819, GNorm = 0.6335, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 0.786416
Epoch 240
Validation binary_cross_entropy = 0.821375
Epoch 241
Validation binary_cross_entropy = 0.879624
Epoch 242
Validation binary_cross_entropy = 0.891403
Epoch 243
Validation binary_cross_entropy = 0.874506
Epoch 244
Loss = 4.4674e-02, PNorm = 67.6467, GNorm = 0.3946, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 0.816510
Epoch 245
Validation binary_cross_entropy = 0.785812
Epoch 246
Validation binary_cross_entropy = 0.822601
Epoch 247
Validation binary_cross_entropy = 0.839148
Epoch 248
Validation binary_cross_entropy = 0.797250
Epoch 249
Loss = 3.1088e-02, PNorm = 67.7285, GNorm = 0.6917, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 0.812645
Epoch 250
Validation binary_cross_entropy = 0.836272
Epoch 251
Validation binary_cross_entropy = 0.905774
Epoch 252
Validation binary_cross_entropy = 0.931950
Epoch 253
Validation binary_cross_entropy = 0.793924
Epoch 254
Loss = 1.0904e-01, PNorm = 67.7938, GNorm = 1.6225, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 0.697587
Epoch 255
Validation binary_cross_entropy = 0.680482
Epoch 256
Validation binary_cross_entropy = 0.684938
Epoch 257
Validation binary_cross_entropy = 0.705648
Epoch 258
Validation binary_cross_entropy = 0.705974
Epoch 259
Loss = 5.6002e-02, PNorm = 67.8455, GNorm = 0.9963, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 0.720226
Epoch 260
Validation binary_cross_entropy = 0.748949
Epoch 261
Validation binary_cross_entropy = 0.790919
Epoch 262
Validation binary_cross_entropy = 0.835619
Epoch 263
Validation binary_cross_entropy = 0.862126
Epoch 264
Loss = 4.5151e-02, PNorm = 67.8979, GNorm = 1.4306, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 0.889121
Epoch 265
Validation binary_cross_entropy = 0.909027
Epoch 266
Validation binary_cross_entropy = 0.931612
Epoch 267
Validation binary_cross_entropy = 0.956948
Epoch 268
Validation binary_cross_entropy = 0.984944
Epoch 269
Loss = 4.6895e-02, PNorm = 67.9426, GNorm = 1.6135, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 1.013820
Epoch 270
Validation binary_cross_entropy = 0.963109
Epoch 271
Validation binary_cross_entropy = 0.990398
Epoch 272
Validation binary_cross_entropy = 0.955213
Epoch 273
Validation binary_cross_entropy = 1.027863
Epoch 274
Loss = 9.8111e-02, PNorm = 67.9907, GNorm = 1.4186, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 1.092378
Epoch 275
Validation binary_cross_entropy = 1.052894
Epoch 276
Validation binary_cross_entropy = 0.980557
Epoch 277
Validation binary_cross_entropy = 0.906990
Epoch 278
Validation binary_cross_entropy = 0.875074
Epoch 279
Loss = 1.0945e-01, PNorm = 68.0396, GNorm = 0.1009, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 0.956885
Epoch 280
Validation binary_cross_entropy = 1.068989
Epoch 281
Validation binary_cross_entropy = 0.979120
Epoch 282
Validation binary_cross_entropy = 0.798740
Epoch 283
Validation binary_cross_entropy = 0.739207
Epoch 284
Loss = 8.1055e-02, PNorm = 68.0874, GNorm = 0.9133, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 0.737170
Epoch 285
Validation binary_cross_entropy = 0.751274
Epoch 286
Validation binary_cross_entropy = 0.768261
Epoch 287
Validation binary_cross_entropy = 0.768341
Epoch 288
Validation binary_cross_entropy = 0.765707
Epoch 289
Loss = 3.9521e-02, PNorm = 68.1373, GNorm = 0.3561, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 0.774728
Epoch 290
Validation binary_cross_entropy = 0.796114
Epoch 291
Validation binary_cross_entropy = 0.830233
Epoch 292
Validation binary_cross_entropy = 0.885370
Epoch 293
Validation binary_cross_entropy = 0.929380
Epoch 294
Loss = 4.8610e-02, PNorm = 68.1756, GNorm = 0.9885, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 0.938708
Epoch 295
Validation binary_cross_entropy = 0.916867
Epoch 296
Validation binary_cross_entropy = 0.910384
Epoch 297
Validation binary_cross_entropy = 0.928141
Epoch 298
Validation binary_cross_entropy = 0.966286
Epoch 299
Loss = 4.0095e-02, PNorm = 68.2091, GNorm = 1.5233, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 1.059284
Model 0 best validation binary_cross_entropy = 0.288250 on epoch 79
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.227363
Ensemble test binary_cross_entropy = 0.227363
Fold 7
Splitting data with seed 7
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.4, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=800, bias=False)
        (W_h): Linear(in_features=800, out_features=800, bias=False)
        (W_o): Linear(in_features=933, out_features=800, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.4, inplace=False)
    (1): Linear(in_features=3100, out_features=800, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.4, inplace=False)
    (4): Linear(in_features=800, out_features=800, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.4, inplace=False)
    (7): Linear(in_features=800, out_features=1, bias=True)
  )
)
Number of parameters = 4,627,201
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 0.287578
Epoch 1
Validation binary_cross_entropy = 0.297397
Epoch 2
Validation binary_cross_entropy = 0.590341
Epoch 3
Validation binary_cross_entropy = 0.310910
Epoch 4
Loss = 9.0203e-01, PNorm = 63.1507, GNorm = 5.0038, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 0.299353
Epoch 5
Validation binary_cross_entropy = 0.396616
Epoch 6
Validation binary_cross_entropy = 0.793094
Epoch 7
Validation binary_cross_entropy = 0.293812
Epoch 8
Validation binary_cross_entropy = 0.344589
Epoch 9
Loss = 7.5659e-01, PNorm = 63.2866, GNorm = 5.1372, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 0.988440
Epoch 10
Validation binary_cross_entropy = 0.341280
Epoch 11
Validation binary_cross_entropy = 0.296808
Epoch 12
Validation binary_cross_entropy = 0.587465
Epoch 13
Validation binary_cross_entropy = 0.562724
Epoch 14
Loss = 7.0592e-01, PNorm = 63.4369, GNorm = 4.6468, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 0.358044
Epoch 15
Validation binary_cross_entropy = 0.408181
Epoch 16
Validation binary_cross_entropy = 0.459692
Epoch 17
Validation binary_cross_entropy = 0.547062
Epoch 18
Validation binary_cross_entropy = 0.555385
Epoch 19
Loss = 6.5802e-01, PNorm = 63.5515, GNorm = 3.5610, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 0.431401
Epoch 20
Validation binary_cross_entropy = 0.434367
Epoch 21
Validation binary_cross_entropy = 0.566763
Epoch 22
Validation binary_cross_entropy = 0.576055
Epoch 23
Validation binary_cross_entropy = 0.375740
Epoch 24
Loss = 6.7032e-01, PNorm = 63.6515, GNorm = 3.1057, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 0.497830
Epoch 25
Validation binary_cross_entropy = 0.461870
Epoch 26
Validation binary_cross_entropy = 0.521824
Epoch 27
Validation binary_cross_entropy = 0.481439
Epoch 28
Validation binary_cross_entropy = 0.457925
Epoch 29
Loss = 6.1084e-01, PNorm = 63.7153, GNorm = 2.8922, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 0.472704
Epoch 30
Validation binary_cross_entropy = 0.517028
Epoch 31
Validation binary_cross_entropy = 0.486613
Epoch 32
Validation binary_cross_entropy = 0.485379
Epoch 33
Validation binary_cross_entropy = 0.468135
Epoch 34
Loss = 5.9811e-01, PNorm = 63.7809, GNorm = 2.4938, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 0.427763
Epoch 35
Validation binary_cross_entropy = 0.469912
Epoch 36
Validation binary_cross_entropy = 0.582947
Epoch 37
Validation binary_cross_entropy = 0.426217
Epoch 38
Validation binary_cross_entropy = 0.360635
Epoch 39
Loss = 6.4190e-01, PNorm = 63.8463, GNorm = 3.5083, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 0.423756
Epoch 40
Validation binary_cross_entropy = 0.641698
Epoch 41
Validation binary_cross_entropy = 0.456837
Epoch 42
Validation binary_cross_entropy = 0.351814
Epoch 43
Validation binary_cross_entropy = 0.423985
Epoch 44
Loss = 5.9507e-01, PNorm = 63.9060, GNorm = 2.3729, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 0.582152
Epoch 45
Validation binary_cross_entropy = 0.512781
Epoch 46
Validation binary_cross_entropy = 0.427940
Epoch 47
Validation binary_cross_entropy = 0.418260
Epoch 48
Validation binary_cross_entropy = 0.491310
Epoch 49
Loss = 6.2303e-01, PNorm = 63.9639, GNorm = 2.3490, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 0.479724
Epoch 50
Validation binary_cross_entropy = 0.441937
Epoch 51
Validation binary_cross_entropy = 0.442449
Epoch 52
Validation binary_cross_entropy = 0.424162
Epoch 53
Validation binary_cross_entropy = 0.467916
Epoch 54
Loss = 5.8621e-01, PNorm = 64.0248, GNorm = 1.7079, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 0.534935
Epoch 55
Validation binary_cross_entropy = 0.497515
Epoch 56
Validation binary_cross_entropy = 0.458755
Epoch 57
Validation binary_cross_entropy = 0.389588
Epoch 58
Validation binary_cross_entropy = 0.409997
Epoch 59
Loss = 5.8892e-01, PNorm = 64.0988, GNorm = 1.9613, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 0.407530
Epoch 60
Validation binary_cross_entropy = 0.524273
Epoch 61
Validation binary_cross_entropy = 0.432197
Epoch 62
Validation binary_cross_entropy = 0.456950
Epoch 63
Validation binary_cross_entropy = 0.508430
Epoch 64
Loss = 5.3164e-01, PNorm = 64.1781, GNorm = 1.6254, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 0.472443
Epoch 65
Validation binary_cross_entropy = 0.359730
Epoch 66
Validation binary_cross_entropy = 0.345241
Epoch 67
Validation binary_cross_entropy = 0.515456
Epoch 68
Validation binary_cross_entropy = 0.433147
Epoch 69
Loss = 4.9257e-01, PNorm = 64.2648, GNorm = 1.4643, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 0.365673
Epoch 70
Validation binary_cross_entropy = 0.412234
Epoch 71
Validation binary_cross_entropy = 0.438998
Epoch 72
Validation binary_cross_entropy = 0.335975
Epoch 73
Validation binary_cross_entropy = 0.418850
Epoch 74
Loss = 4.5810e-01, PNorm = 64.3710, GNorm = 1.7875, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 0.437598
Epoch 75
Validation binary_cross_entropy = 0.341689
Epoch 76
Validation binary_cross_entropy = 0.415183
Epoch 77
Validation binary_cross_entropy = 0.423582
Epoch 78
Validation binary_cross_entropy = 0.330176
Epoch 79
Loss = 4.6943e-01, PNorm = 64.4765, GNorm = 1.8497, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 0.450325
Epoch 80
Validation binary_cross_entropy = 0.415297
Epoch 81
Validation binary_cross_entropy = 0.322829
Epoch 82
Validation binary_cross_entropy = 0.406414
Epoch 83
Validation binary_cross_entropy = 0.519345
Epoch 84
Loss = 5.1618e-01, PNorm = 64.5854, GNorm = 1.7793, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 0.313922
Epoch 85
Validation binary_cross_entropy = 0.443533
Epoch 86
Validation binary_cross_entropy = 0.518932
Epoch 87
Validation binary_cross_entropy = 0.403173
Epoch 88
Validation binary_cross_entropy = 0.378945
Epoch 89
Loss = 4.0232e-01, PNorm = 64.7009, GNorm = 1.8410, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 0.460353
Epoch 90
Validation binary_cross_entropy = 0.357802
Epoch 91
Validation binary_cross_entropy = 0.298252
Epoch 92
Validation binary_cross_entropy = 0.688198
Epoch 93
Validation binary_cross_entropy = 0.446243
Epoch 94
Loss = 3.8489e-01, PNorm = 64.8121, GNorm = 1.4452, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 0.321528
Epoch 95
Validation binary_cross_entropy = 0.336616
Epoch 96
Validation binary_cross_entropy = 0.614482
Epoch 97
Validation binary_cross_entropy = 0.421732
Epoch 98
Validation binary_cross_entropy = 0.309133
Epoch 99
Loss = 3.8713e-01, PNorm = 64.9433, GNorm = 1.7246, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.412696
Epoch 100
Validation binary_cross_entropy = 0.487821
Epoch 101
Validation binary_cross_entropy = 0.368465
Epoch 102
Validation binary_cross_entropy = 0.354444
Epoch 103
Validation binary_cross_entropy = 0.438060
Epoch 104
Loss = 3.3578e-01, PNorm = 65.0739, GNorm = 1.5572, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.409859
Epoch 105
Validation binary_cross_entropy = 0.347598
Epoch 106
Validation binary_cross_entropy = 0.414044
Epoch 107
Validation binary_cross_entropy = 0.397244
Epoch 108
Validation binary_cross_entropy = 0.334173
Epoch 109
Loss = 3.3298e-01, PNorm = 65.1985, GNorm = 2.6815, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.513396
Epoch 110
Validation binary_cross_entropy = 0.483032
Epoch 111
Validation binary_cross_entropy = 0.370771
Epoch 112
Validation binary_cross_entropy = 0.475437
Epoch 113
Validation binary_cross_entropy = 0.522656
Epoch 114
Loss = 3.4429e-01, PNorm = 65.3315, GNorm = 1.3007, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.379143
Epoch 115
Validation binary_cross_entropy = 0.430793
Epoch 116
Validation binary_cross_entropy = 0.527545
Epoch 117
Validation binary_cross_entropy = 0.359518
Epoch 118
Validation binary_cross_entropy = 0.321336
Epoch 119
Loss = 4.1713e-01, PNorm = 65.4728, GNorm = 1.4663, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.378599
Epoch 120
Validation binary_cross_entropy = 0.492115
Epoch 121
Validation binary_cross_entropy = 0.388650
Epoch 122
Validation binary_cross_entropy = 0.424830
Epoch 123
Validation binary_cross_entropy = 0.495568
Epoch 124
Loss = 2.6186e-01, PNorm = 65.6155, GNorm = 1.0727, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.512523
Epoch 125
Validation binary_cross_entropy = 0.550059
Epoch 126
Validation binary_cross_entropy = 0.543183
Epoch 127
Validation binary_cross_entropy = 0.542042
Epoch 128
Validation binary_cross_entropy = 0.507629
Epoch 129
Loss = 1.7519e-01, PNorm = 65.7447, GNorm = 1.4237, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.488613
Epoch 130
Validation binary_cross_entropy = 0.484101
Epoch 131
Validation binary_cross_entropy = 0.480331
Epoch 132
Validation binary_cross_entropy = 0.526641
Epoch 133
Validation binary_cross_entropy = 0.642370
Epoch 134
Loss = 2.8842e-01, PNorm = 65.8746, GNorm = 3.5129, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.527713
Epoch 135
Validation binary_cross_entropy = 0.502124
Epoch 136
Validation binary_cross_entropy = 0.554298
Epoch 137
Validation binary_cross_entropy = 0.489662
Epoch 138
Validation binary_cross_entropy = 0.496444
Epoch 139
Loss = 2.2473e-01, PNorm = 66.0105, GNorm = 1.1665, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.578672
Epoch 140
Validation binary_cross_entropy = 0.674068
Epoch 141
Validation binary_cross_entropy = 0.579720
Epoch 142
Validation binary_cross_entropy = 0.600954
Epoch 143
Validation binary_cross_entropy = 0.671611
Epoch 144
Loss = 1.8399e-01, PNorm = 66.1370, GNorm = 1.9431, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.705063
Epoch 145
Validation binary_cross_entropy = 0.745262
Epoch 146
Validation binary_cross_entropy = 0.791273
Epoch 147
Validation binary_cross_entropy = 0.786166
Epoch 148
Validation binary_cross_entropy = 0.715535
Epoch 149
Loss = 1.6662e-01, PNorm = 66.2380, GNorm = 3.7739, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.675608
Epoch 150
Validation binary_cross_entropy = 0.587579
Epoch 151
Validation binary_cross_entropy = 0.535914
Epoch 152
Validation binary_cross_entropy = 0.518489
Epoch 153
Validation binary_cross_entropy = 0.515610
Epoch 154
Loss = 1.7910e-01, PNorm = 66.3425, GNorm = 1.7638, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.554122
Epoch 155
Validation binary_cross_entropy = 0.570995
Epoch 156
Validation binary_cross_entropy = 0.605536
Epoch 157
Validation binary_cross_entropy = 0.608824
Epoch 158
Validation binary_cross_entropy = 0.664970
Epoch 159
Loss = 1.8547e-01, PNorm = 66.4384, GNorm = 3.0301, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.659517
Epoch 160
Validation binary_cross_entropy = 0.678668
Epoch 161
Validation binary_cross_entropy = 0.698399
Epoch 162
Validation binary_cross_entropy = 0.696986
Epoch 163
Validation binary_cross_entropy = 0.682270
Epoch 164
Loss = 1.3554e-01, PNorm = 66.5338, GNorm = 3.5417, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.699160
Epoch 165
Validation binary_cross_entropy = 0.759140
Epoch 166
Validation binary_cross_entropy = 0.697140
Epoch 167
Validation binary_cross_entropy = 0.667196
Epoch 168
Validation binary_cross_entropy = 0.706802
Epoch 169
Loss = 1.8584e-01, PNorm = 66.6263, GNorm = 0.6898, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.774440
Epoch 170
Validation binary_cross_entropy = 0.774215
Epoch 171
Validation binary_cross_entropy = 0.745034
Epoch 172
Validation binary_cross_entropy = 0.737186
Epoch 173
Validation binary_cross_entropy = 0.786655
Epoch 174
Loss = 1.4078e-01, PNorm = 66.7301, GNorm = 1.3500, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 0.806647
Epoch 175
Validation binary_cross_entropy = 0.800242
Epoch 176
Validation binary_cross_entropy = 0.921135
Epoch 177
Validation binary_cross_entropy = 0.701197
Epoch 178
Validation binary_cross_entropy = 0.594028
Epoch 179
Loss = 1.3529e-01, PNorm = 66.8368, GNorm = 1.0233, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 0.568053
Epoch 180
Validation binary_cross_entropy = 0.536981
Epoch 181
Validation binary_cross_entropy = 0.557621
Epoch 182
Validation binary_cross_entropy = 0.596732
Epoch 183
Validation binary_cross_entropy = 0.630400
Epoch 184
Loss = 9.7319e-02, PNorm = 66.9607, GNorm = 1.3460, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 0.683154
Epoch 185
Validation binary_cross_entropy = 0.720070
Epoch 186
Validation binary_cross_entropy = 0.752898
Epoch 187
Validation binary_cross_entropy = 0.803656
Epoch 188
Validation binary_cross_entropy = 0.788732
Epoch 189
Loss = 6.4205e-02, PNorm = 67.0530, GNorm = 1.7694, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 0.864521
Epoch 190
Validation binary_cross_entropy = 0.801609
Epoch 191
Validation binary_cross_entropy = 0.835918
Epoch 192
Validation binary_cross_entropy = 0.799484
Epoch 193
Validation binary_cross_entropy = 0.710151
Epoch 194
Loss = 1.2747e-01, PNorm = 67.1372, GNorm = 2.8043, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 0.713651
Epoch 195
Validation binary_cross_entropy = 0.671399
Epoch 196
Validation binary_cross_entropy = 0.731035
Epoch 197
Validation binary_cross_entropy = 0.731456
Epoch 198
Validation binary_cross_entropy = 0.703867
Epoch 199
Loss = 2.2353e-01, PNorm = 67.2191, GNorm = 4.1797, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 0.735032
Epoch 200
Validation binary_cross_entropy = 0.875234
Epoch 201
Validation binary_cross_entropy = 0.845570
Epoch 202
Validation binary_cross_entropy = 0.729508
Epoch 203
Validation binary_cross_entropy = 0.694990
Epoch 204
Loss = 1.1624e-01, PNorm = 67.3057, GNorm = 1.7970, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 0.709775
Epoch 205
Validation binary_cross_entropy = 0.673603
Epoch 206
Validation binary_cross_entropy = 0.686100
Epoch 207
Validation binary_cross_entropy = 0.640591
Epoch 208
Validation binary_cross_entropy = 0.622091
Epoch 209
Loss = 8.8846e-02, PNorm = 67.3843, GNorm = 1.1637, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 0.629825
Epoch 210
Validation binary_cross_entropy = 0.669578
Epoch 211
Validation binary_cross_entropy = 0.714685
Epoch 212
Validation binary_cross_entropy = 0.712062
Epoch 213
Validation binary_cross_entropy = 0.733969
Epoch 214
Loss = 1.2771e-01, PNorm = 67.4633, GNorm = 0.7641, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 0.775567
Epoch 215
Validation binary_cross_entropy = 0.808647
Epoch 216
Validation binary_cross_entropy = 0.854152
Epoch 217
Validation binary_cross_entropy = 0.845440
Epoch 218
Validation binary_cross_entropy = 0.766352
Epoch 219
Loss = 6.2457e-02, PNorm = 67.5301, GNorm = 1.2830, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 0.751419
Epoch 220
Validation binary_cross_entropy = 0.756421
Epoch 221
Validation binary_cross_entropy = 0.770285
Epoch 222
Validation binary_cross_entropy = 0.796439
Epoch 223
Validation binary_cross_entropy = 0.810559
Epoch 224
Loss = 6.6946e-02, PNorm = 67.5919, GNorm = 0.7677, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 0.821371
Epoch 225
Validation binary_cross_entropy = 0.856219
Epoch 226
Validation binary_cross_entropy = 0.921635
Epoch 227
Validation binary_cross_entropy = 0.988436
Epoch 228
Validation binary_cross_entropy = 1.015962
Epoch 229
Loss = 5.4501e-02, PNorm = 67.6420, GNorm = 1.6215, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 1.064670
Epoch 230
Validation binary_cross_entropy = 1.100842
Epoch 231
Validation binary_cross_entropy = 1.110163
Epoch 232
Validation binary_cross_entropy = 1.103050
Epoch 233
Validation binary_cross_entropy = 1.117529
Epoch 234
Loss = 7.8469e-02, PNorm = 67.6957, GNorm = 2.0957, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 0.979556
Epoch 235
Validation binary_cross_entropy = 0.891325
Epoch 236
Validation binary_cross_entropy = 0.824478
Epoch 237
Validation binary_cross_entropy = 0.802508
Epoch 238
Validation binary_cross_entropy = 0.815035
Epoch 239
Loss = 1.4276e-01, PNorm = 67.7460, GNorm = 1.5064, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 0.753825
Epoch 240
Validation binary_cross_entropy = 0.703224
Epoch 241
Validation binary_cross_entropy = 0.715579
Epoch 242
Validation binary_cross_entropy = 0.769384
Epoch 243
Validation binary_cross_entropy = 0.816311
Epoch 244
Loss = 9.6183e-02, PNorm = 67.8153, GNorm = 0.8879, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 0.879800
Epoch 245
Validation binary_cross_entropy = 0.951524
Epoch 246
Validation binary_cross_entropy = 0.981683
Epoch 247
Validation binary_cross_entropy = 1.051914
Epoch 248
Validation binary_cross_entropy = 1.095258
Epoch 249
Loss = 4.7169e-02, PNorm = 67.8732, GNorm = 0.4464, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 1.024466
Epoch 250
Validation binary_cross_entropy = 1.004154
Epoch 251
Validation binary_cross_entropy = 0.967364
Epoch 252
Validation binary_cross_entropy = 0.908230
Epoch 253
Validation binary_cross_entropy = 0.852860
Epoch 254
Loss = 6.3485e-02, PNorm = 67.9236, GNorm = 1.0660, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 0.820879
Epoch 255
Validation binary_cross_entropy = 0.795674
Epoch 256
Validation binary_cross_entropy = 0.789801
Epoch 257
Validation binary_cross_entropy = 0.793679
Epoch 258
Validation binary_cross_entropy = 0.820466
Epoch 259
Loss = 9.9206e-02, PNorm = 67.9862, GNorm = 2.0106, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 0.887989
Epoch 260
Validation binary_cross_entropy = 0.916336
Epoch 261
Validation binary_cross_entropy = 0.916691
Epoch 262
Validation binary_cross_entropy = 0.925399
Epoch 263
Validation binary_cross_entropy = 0.965837
Epoch 264
Loss = 1.1548e-01, PNorm = 68.0462, GNorm = 3.4384, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 0.973522
Epoch 265
Validation binary_cross_entropy = 1.023698
Epoch 266
Validation binary_cross_entropy = 1.039478
Epoch 267
Validation binary_cross_entropy = 1.005325
Epoch 268
Validation binary_cross_entropy = 1.015441
Epoch 269
Loss = 6.5457e-02, PNorm = 68.0935, GNorm = 1.0933, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 1.009015
Epoch 270
Validation binary_cross_entropy = 1.012466
Epoch 271
Validation binary_cross_entropy = 1.006424
Epoch 272
Validation binary_cross_entropy = 0.961708
Epoch 273
Validation binary_cross_entropy = 0.943437
Epoch 274
Loss = 6.7687e-02, PNorm = 68.1480, GNorm = 0.8594, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 0.930276
Epoch 275
Validation binary_cross_entropy = 0.929971
Epoch 276
Validation binary_cross_entropy = 0.924498
Epoch 277
Validation binary_cross_entropy = 0.926440
Epoch 278
Validation binary_cross_entropy = 0.926770
Epoch 279
Loss = 6.4921e-02, PNorm = 68.2038, GNorm = 1.6578, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 1.014007
Epoch 280
Validation binary_cross_entropy = 0.942076
Epoch 281
Validation binary_cross_entropy = 1.022825
Epoch 282
Validation binary_cross_entropy = 0.938791
Epoch 283
Validation binary_cross_entropy = 0.919641
Epoch 284
Loss = 8.2633e-02, PNorm = 68.2679, GNorm = 2.9245, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 0.880076
Epoch 285
Validation binary_cross_entropy = 0.867580
Epoch 286
Validation binary_cross_entropy = 0.888576
Epoch 287
Validation binary_cross_entropy = 0.794869
Epoch 288
Validation binary_cross_entropy = 0.758853
Epoch 289
Loss = 8.0506e-02, PNorm = 68.3306, GNorm = 0.9081, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 0.747756
Epoch 290
Validation binary_cross_entropy = 0.727784
Epoch 291
Validation binary_cross_entropy = 0.755987
Epoch 292
Validation binary_cross_entropy = 0.776505
Epoch 293
Validation binary_cross_entropy = 0.787216
Epoch 294
Loss = 2.4763e-02, PNorm = 68.3990, GNorm = 0.8119, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 0.834675
Epoch 295
Validation binary_cross_entropy = 0.865502
Epoch 296
Validation binary_cross_entropy = 0.914333
Epoch 297
Validation binary_cross_entropy = 0.958716
Epoch 298
Validation binary_cross_entropy = 0.988244
Epoch 299
Loss = 5.5585e-02, PNorm = 68.4531, GNorm = 0.7104, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 0.999965
Model 0 best validation binary_cross_entropy = 0.287578 on epoch 0
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.223729
Ensemble test binary_cross_entropy = 0.223729
Fold 8
Splitting data with seed 8
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.4, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=800, bias=False)
        (W_h): Linear(in_features=800, out_features=800, bias=False)
        (W_o): Linear(in_features=933, out_features=800, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.4, inplace=False)
    (1): Linear(in_features=3100, out_features=800, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.4, inplace=False)
    (4): Linear(in_features=800, out_features=800, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.4, inplace=False)
    (7): Linear(in_features=800, out_features=1, bias=True)
  )
)
Number of parameters = 4,627,201
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 0.287546
Epoch 1
Validation binary_cross_entropy = 0.301045
Epoch 2
Validation binary_cross_entropy = 0.583419
Epoch 3
Validation binary_cross_entropy = 0.319260
Epoch 4
Loss = 8.4577e-01, PNorm = 63.1524, GNorm = 7.6351, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 0.297064
Epoch 5
Validation binary_cross_entropy = 0.525440
Epoch 6
Validation binary_cross_entropy = 0.813832
Epoch 7
Validation binary_cross_entropy = 0.322716
Epoch 8
Validation binary_cross_entropy = 0.430601
Epoch 9
Loss = 6.5036e-01, PNorm = 63.2942, GNorm = 4.3828, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 0.589930
Epoch 10
Validation binary_cross_entropy = 0.347977
Epoch 11
Validation binary_cross_entropy = 0.523647
Epoch 12
Validation binary_cross_entropy = 0.731850
Epoch 13
Validation binary_cross_entropy = 0.302254
Epoch 14
Loss = 9.3429e-01, PNorm = 63.4532, GNorm = 6.9298, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 0.343261
Epoch 15
Validation binary_cross_entropy = 0.570982
Epoch 16
Validation binary_cross_entropy = 0.376336
Epoch 17
Validation binary_cross_entropy = 0.437099
Epoch 18
Validation binary_cross_entropy = 0.617986
Epoch 19
Loss = 7.1439e-01, PNorm = 63.5767, GNorm = 3.5800, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 0.366876
Epoch 20
Validation binary_cross_entropy = 0.333799
Epoch 21
Validation binary_cross_entropy = 0.654548
Epoch 22
Validation binary_cross_entropy = 0.631589
Epoch 23
Validation binary_cross_entropy = 0.338761
Epoch 24
Loss = 7.0166e-01, PNorm = 63.6766, GNorm = 4.7686, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 0.346973
Epoch 25
Validation binary_cross_entropy = 0.618859
Epoch 26
Validation binary_cross_entropy = 0.695850
Epoch 27
Validation binary_cross_entropy = 0.450053
Epoch 28
Validation binary_cross_entropy = 0.402421
Epoch 29
Loss = 5.9487e-01, PNorm = 63.7513, GNorm = 2.6708, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 0.571627
Epoch 30
Validation binary_cross_entropy = 0.708425
Epoch 31
Validation binary_cross_entropy = 0.513279
Epoch 32
Validation binary_cross_entropy = 0.401880
Epoch 33
Validation binary_cross_entropy = 0.388981
Epoch 34
Loss = 6.0567e-01, PNorm = 63.8215, GNorm = 3.1519, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 0.544627
Epoch 35
Validation binary_cross_entropy = 0.651220
Epoch 36
Validation binary_cross_entropy = 0.560333
Epoch 37
Validation binary_cross_entropy = 0.471793
Epoch 38
Validation binary_cross_entropy = 0.491366
Epoch 39
Loss = 6.4089e-01, PNorm = 63.8817, GNorm = 2.5876, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 0.584887
Epoch 40
Validation binary_cross_entropy = 0.602236
Epoch 41
Validation binary_cross_entropy = 0.543659
Epoch 42
Validation binary_cross_entropy = 0.472970
Epoch 43
Validation binary_cross_entropy = 0.478900
Epoch 44
Loss = 6.5011e-01, PNorm = 63.9424, GNorm = 2.2508, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 0.502020
Epoch 45
Validation binary_cross_entropy = 0.531290
Epoch 46
Validation binary_cross_entropy = 0.605189
Epoch 47
Validation binary_cross_entropy = 0.578180
Epoch 48
Validation binary_cross_entropy = 0.472469
Epoch 49
Loss = 5.6602e-01, PNorm = 64.0065, GNorm = 2.1797, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 0.445336
Epoch 50
Validation binary_cross_entropy = 0.601404
Epoch 51
Validation binary_cross_entropy = 0.527437
Epoch 52
Validation binary_cross_entropy = 0.444580
Epoch 53
Validation binary_cross_entropy = 0.518978
Epoch 54
Loss = 5.0561e-01, PNorm = 64.0863, GNorm = 2.3914, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 0.513229
Epoch 55
Validation binary_cross_entropy = 0.416871
Epoch 56
Validation binary_cross_entropy = 0.385323
Epoch 57
Validation binary_cross_entropy = 0.439685
Epoch 58
Validation binary_cross_entropy = 0.477131
Epoch 59
Loss = 5.2129e-01, PNorm = 64.1718, GNorm = 1.8437, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 0.375356
Epoch 60
Validation binary_cross_entropy = 0.405288
Epoch 61
Validation binary_cross_entropy = 0.500979
Epoch 62
Validation binary_cross_entropy = 0.373644
Epoch 63
Validation binary_cross_entropy = 0.492720
Epoch 64
Loss = 4.4872e-01, PNorm = 64.2654, GNorm = 1.8776, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 0.475682
Epoch 65
Validation binary_cross_entropy = 0.407719
Epoch 66
Validation binary_cross_entropy = 0.432721
Epoch 67
Validation binary_cross_entropy = 0.347950
Epoch 68
Validation binary_cross_entropy = 0.510376
Epoch 69
Loss = 5.1953e-01, PNorm = 64.3588, GNorm = 1.8400, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 0.306464
Epoch 70
Validation binary_cross_entropy = 0.471980
Epoch 71
Validation binary_cross_entropy = 0.507622
Epoch 72
Validation binary_cross_entropy = 0.364043
Epoch 73
Validation binary_cross_entropy = 0.382895
Epoch 74
Loss = 4.3330e-01, PNorm = 64.4643, GNorm = 2.4359, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 0.396784
Epoch 75
Validation binary_cross_entropy = 0.327292
Epoch 76
Validation binary_cross_entropy = 0.387187
Epoch 77
Validation binary_cross_entropy = 0.335374
Epoch 78
Validation binary_cross_entropy = 0.425642
Epoch 79
Loss = 4.3273e-01, PNorm = 64.5663, GNorm = 1.8838, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 0.455353
Epoch 80
Validation binary_cross_entropy = 0.370196
Epoch 81
Validation binary_cross_entropy = 0.391624
Epoch 82
Validation binary_cross_entropy = 0.372607
Epoch 83
Validation binary_cross_entropy = 0.331110
Epoch 84
Loss = 3.4628e-01, PNorm = 64.6883, GNorm = 1.7338, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 0.508530
Epoch 85
Validation binary_cross_entropy = 0.338244
Epoch 86
Validation binary_cross_entropy = 0.408943
Epoch 87
Validation binary_cross_entropy = 0.457768
Epoch 88
Validation binary_cross_entropy = 0.318494
Epoch 89
Loss = 4.5395e-01, PNorm = 64.8013, GNorm = 2.4420, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 0.482693
Epoch 90
Validation binary_cross_entropy = 0.375504
Epoch 91
Validation binary_cross_entropy = 0.310761
Epoch 92
Validation binary_cross_entropy = 0.521798
Epoch 93
Validation binary_cross_entropy = 0.321695
Epoch 94
Loss = 4.8637e-01, PNorm = 64.9364, GNorm = 4.4356, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 0.308631
Epoch 95
Validation binary_cross_entropy = 0.458235
Epoch 96
Validation binary_cross_entropy = 0.425203
Epoch 97
Validation binary_cross_entropy = 0.341972
Epoch 98
Validation binary_cross_entropy = 0.410270
Epoch 99
Loss = 3.4311e-01, PNorm = 65.0810, GNorm = 1.9150, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.517458
Epoch 100
Validation binary_cross_entropy = 0.347972
Epoch 101
Validation binary_cross_entropy = 0.395681
Epoch 102
Validation binary_cross_entropy = 0.401147
Epoch 103
Validation binary_cross_entropy = 0.346039
Epoch 104
Loss = 2.8668e-01, PNorm = 65.2046, GNorm = 1.5033, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.396467
Epoch 105
Validation binary_cross_entropy = 0.425795
Epoch 106
Validation binary_cross_entropy = 0.441654
Epoch 107
Validation binary_cross_entropy = 0.519421
Epoch 108
Validation binary_cross_entropy = 0.465037
Epoch 109
Loss = 2.7748e-01, PNorm = 65.3302, GNorm = 2.1324, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.522512
Epoch 110
Validation binary_cross_entropy = 0.335623
Epoch 111
Validation binary_cross_entropy = 0.290490
Epoch 112
Validation binary_cross_entropy = 0.365171
Epoch 113
Validation binary_cross_entropy = 0.524182
Epoch 114
Loss = 5.0223e-01, PNorm = 65.4578, GNorm = 1.6733, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.336200
Epoch 115
Validation binary_cross_entropy = 0.297966
Epoch 116
Validation binary_cross_entropy = 0.339121
Epoch 117
Validation binary_cross_entropy = 0.435102
Epoch 118
Validation binary_cross_entropy = 0.436798
Epoch 119
Loss = 2.9040e-01, PNorm = 65.5984, GNorm = 2.1362, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.489594
Epoch 120
Validation binary_cross_entropy = 0.499471
Epoch 121
Validation binary_cross_entropy = 0.494762
Epoch 122
Validation binary_cross_entropy = 0.456593
Epoch 123
Validation binary_cross_entropy = 0.440197
Epoch 124
Loss = 2.0595e-01, PNorm = 65.7118, GNorm = 1.5369, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.405393
Epoch 125
Validation binary_cross_entropy = 0.411530
Epoch 126
Validation binary_cross_entropy = 0.423887
Epoch 127
Validation binary_cross_entropy = 0.446198
Epoch 128
Validation binary_cross_entropy = 0.481814
Epoch 129
Loss = 1.4175e-01, PNorm = 65.8289, GNorm = 1.7824, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.521217
Epoch 130
Validation binary_cross_entropy = 0.537195
Epoch 131
Validation binary_cross_entropy = 0.512177
Epoch 132
Validation binary_cross_entropy = 0.504164
Epoch 133
Validation binary_cross_entropy = 0.484348
Epoch 134
Loss = 1.5122e-01, PNorm = 65.9322, GNorm = 2.6983, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.498956
Epoch 135
Validation binary_cross_entropy = 0.435979
Epoch 136
Validation binary_cross_entropy = 0.430129
Epoch 137
Validation binary_cross_entropy = 0.484654
Epoch 138
Validation binary_cross_entropy = 0.455105
Epoch 139
Loss = 1.8520e-01, PNorm = 66.0361, GNorm = 5.7245, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.463649
Epoch 140
Validation binary_cross_entropy = 0.518297
Epoch 141
Validation binary_cross_entropy = 0.485850
Epoch 142
Validation binary_cross_entropy = 0.527778
Epoch 143
Validation binary_cross_entropy = 0.513994
Epoch 144
Loss = 1.9564e-01, PNorm = 66.1431, GNorm = 2.5383, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.617826
Epoch 145
Validation binary_cross_entropy = 0.521381
Epoch 146
Validation binary_cross_entropy = 0.532412
Epoch 147
Validation binary_cross_entropy = 0.519980
Epoch 148
Validation binary_cross_entropy = 0.495770
Epoch 149
Loss = 1.7205e-01, PNorm = 66.2447, GNorm = 0.9299, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.506710
Epoch 150
Validation binary_cross_entropy = 0.531751
Epoch 151
Validation binary_cross_entropy = 0.569683
Epoch 152
Validation binary_cross_entropy = 0.585215
Epoch 153
Validation binary_cross_entropy = 0.612885
Epoch 154
Loss = 1.2968e-01, PNorm = 66.3352, GNorm = 2.6267, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.620807
Epoch 155
Validation binary_cross_entropy = 0.623851
Epoch 156
Validation binary_cross_entropy = 0.637385
Epoch 157
Validation binary_cross_entropy = 0.650803
Epoch 158
Validation binary_cross_entropy = 0.676008
Epoch 159
Loss = 1.1395e-01, PNorm = 66.4153, GNorm = 1.3483, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.712153
Epoch 160
Validation binary_cross_entropy = 0.715740
Epoch 161
Validation binary_cross_entropy = 0.700757
Epoch 162
Validation binary_cross_entropy = 0.693427
Epoch 163
Validation binary_cross_entropy = 0.689762
Epoch 164
Loss = 1.3241e-01, PNorm = 66.4816, GNorm = 1.3883, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.665492
Epoch 165
Validation binary_cross_entropy = 0.640673
Epoch 166
Validation binary_cross_entropy = 0.649005
Epoch 167
Validation binary_cross_entropy = 0.670653
Epoch 168
Validation binary_cross_entropy = 0.688505
Epoch 169
Loss = 1.4489e-01, PNorm = 66.5509, GNorm = 1.3800, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.697485
Epoch 170
Validation binary_cross_entropy = 0.699114
Epoch 171
Validation binary_cross_entropy = 0.713904
Epoch 172
Validation binary_cross_entropy = 0.726096
Epoch 173
Validation binary_cross_entropy = 0.746595
Epoch 174
Loss = 7.4059e-02, PNorm = 66.6248, GNorm = 0.9997, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 0.812994
Epoch 175
Validation binary_cross_entropy = 0.866543
Epoch 176
Validation binary_cross_entropy = 0.880532
Epoch 177
Validation binary_cross_entropy = 0.920976
Epoch 178
Validation binary_cross_entropy = 0.941716
Epoch 179
Loss = 9.5584e-02, PNorm = 66.6998, GNorm = 1.1224, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 0.953082
Epoch 180
Validation binary_cross_entropy = 1.002095
Epoch 181
Validation binary_cross_entropy = 0.979125
Epoch 182
Validation binary_cross_entropy = 0.886137
Epoch 183
Validation binary_cross_entropy = 0.795813
Epoch 184
Loss = 9.4787e-02, PNorm = 66.7664, GNorm = 3.9095, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 0.780428
Epoch 185
Validation binary_cross_entropy = 0.774053
Epoch 186
Validation binary_cross_entropy = 0.747493
Epoch 187
Validation binary_cross_entropy = 0.785110
Epoch 188
Validation binary_cross_entropy = 0.797144
Epoch 189
Loss = 1.6604e-01, PNorm = 66.8575, GNorm = 1.0933, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 0.812701
Epoch 190
Validation binary_cross_entropy = 0.793171
Epoch 191
Validation binary_cross_entropy = 0.781198
Epoch 192
Validation binary_cross_entropy = 0.787704
Epoch 193
Validation binary_cross_entropy = 0.864741
Epoch 194
Loss = 1.0508e-01, PNorm = 66.9352, GNorm = 1.1071, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 0.841284
Epoch 195
Validation binary_cross_entropy = 0.828268
Epoch 196
Validation binary_cross_entropy = 0.821455
Epoch 197
Validation binary_cross_entropy = 0.849786
Epoch 198
Validation binary_cross_entropy = 0.828047
Epoch 199
Loss = 1.5126e-01, PNorm = 66.9967, GNorm = 1.6932, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 0.860740
Epoch 200
Validation binary_cross_entropy = 0.840062
Epoch 201
Validation binary_cross_entropy = 0.823786
Epoch 202
Validation binary_cross_entropy = 0.836618
Epoch 203
Validation binary_cross_entropy = 0.847698
Epoch 204
Loss = 9.2348e-02, PNorm = 67.0542, GNorm = 3.7293, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 0.810633
Epoch 205
Validation binary_cross_entropy = 0.788968
Epoch 206
Validation binary_cross_entropy = 0.811492
Epoch 207
Validation binary_cross_entropy = 0.827364
Epoch 208
Validation binary_cross_entropy = 0.896396
Epoch 209
Loss = 5.1363e-02, PNorm = 67.1177, GNorm = 1.5731, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 0.979551
Epoch 210
Validation binary_cross_entropy = 0.998246
Epoch 211
Validation binary_cross_entropy = 1.003611
Epoch 212
Validation binary_cross_entropy = 1.023584
Epoch 213
Validation binary_cross_entropy = 0.996808
Epoch 214
Loss = 3.8779e-02, PNorm = 67.1723, GNorm = 1.7164, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 0.994138
Epoch 215
Validation binary_cross_entropy = 0.992382
Epoch 216
Validation binary_cross_entropy = 1.032755
Epoch 217
Validation binary_cross_entropy = 1.046893
Epoch 218
Validation binary_cross_entropy = 1.087404
Epoch 219
Loss = 4.1366e-02, PNorm = 67.2220, GNorm = 0.8613, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 1.198201
Epoch 220
Validation binary_cross_entropy = 1.146503
Epoch 221
Validation binary_cross_entropy = 1.096649
Epoch 222
Validation binary_cross_entropy = 1.004526
Epoch 223
Validation binary_cross_entropy = 1.060539
Epoch 224
Loss = 1.5561e-01, PNorm = 67.2725, GNorm = 5.9531, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 0.984987
Epoch 225
Validation binary_cross_entropy = 0.860323
Epoch 226
Validation binary_cross_entropy = 0.796541
Epoch 227
Validation binary_cross_entropy = 0.799652
Epoch 228
Validation binary_cross_entropy = 0.830485
Epoch 229
Loss = 6.5051e-02, PNorm = 67.3385, GNorm = 0.8570, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 0.756934
Epoch 230
Validation binary_cross_entropy = 0.734597
Epoch 231
Validation binary_cross_entropy = 0.750151
Epoch 232
Validation binary_cross_entropy = 0.853515
Epoch 233
Validation binary_cross_entropy = 0.899295
Epoch 234
Loss = 5.9383e-02, PNorm = 67.4004, GNorm = 2.1420, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 0.889633
Epoch 235
Validation binary_cross_entropy = 0.933429
Epoch 236
Validation binary_cross_entropy = 0.978365
Epoch 237
Validation binary_cross_entropy = 1.199200
Epoch 238
Validation binary_cross_entropy = 0.878247
Epoch 239
Loss = 1.9607e-01, PNorm = 67.4600, GNorm = 2.8993, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 0.767469
Epoch 240
Validation binary_cross_entropy = 0.590216
Epoch 241
Validation binary_cross_entropy = 0.523086
Epoch 242
Validation binary_cross_entropy = 0.492568
Epoch 243
Validation binary_cross_entropy = 0.475630
Epoch 244
Loss = 1.9466e-01, PNorm = 67.5315, GNorm = 1.4084, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 0.509687
Epoch 245
Validation binary_cross_entropy = 0.555092
Epoch 246
Validation binary_cross_entropy = 0.585116
Epoch 247
Validation binary_cross_entropy = 0.716313
Epoch 248
Validation binary_cross_entropy = 0.831888
Epoch 249
Loss = 1.8624e-01, PNorm = 67.6102, GNorm = 4.6866, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 0.799740
Epoch 250
Validation binary_cross_entropy = 0.766069
Epoch 251
Validation binary_cross_entropy = 0.798226
Epoch 252
Validation binary_cross_entropy = 0.828309
Epoch 253
Validation binary_cross_entropy = 0.880801
Epoch 254
Loss = 7.7348e-02, PNorm = 67.6752, GNorm = 2.4505, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 0.941758
Epoch 255
Validation binary_cross_entropy = 0.903260
Epoch 256
Validation binary_cross_entropy = 0.894290
Epoch 257
Validation binary_cross_entropy = 0.906302
Epoch 258
Validation binary_cross_entropy = 0.935907
Epoch 259
Loss = 6.8140e-02, PNorm = 67.7337, GNorm = 1.5150, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 1.007627
Epoch 260
Validation binary_cross_entropy = 0.970751
Epoch 261
Validation binary_cross_entropy = 0.899228
Epoch 262
Validation binary_cross_entropy = 0.891010
Epoch 263
Validation binary_cross_entropy = 0.901733
Epoch 264
Loss = 8.8919e-02, PNorm = 67.7765, GNorm = 1.5320, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 0.913651
Epoch 265
Validation binary_cross_entropy = 0.864576
Epoch 266
Validation binary_cross_entropy = 0.867519
Epoch 267
Validation binary_cross_entropy = 0.886764
Epoch 268
Validation binary_cross_entropy = 0.912198
Epoch 269
Loss = 4.6500e-02, PNorm = 67.8294, GNorm = 0.6394, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 0.983108
Epoch 270
Validation binary_cross_entropy = 1.062547
Epoch 271
Validation binary_cross_entropy = 1.063928
Epoch 272
Validation binary_cross_entropy = 1.075137
Epoch 273
Validation binary_cross_entropy = 1.145001
Epoch 274
Loss = 8.5132e-02, PNorm = 67.8737, GNorm = 1.5733, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 1.094972
Epoch 275
Validation binary_cross_entropy = 1.053515
Epoch 276
Validation binary_cross_entropy = 1.049752
Epoch 277
Validation binary_cross_entropy = 1.052662
Epoch 278
Validation binary_cross_entropy = 1.061819
Epoch 279
Loss = 1.4937e-02, PNorm = 67.9177, GNorm = 0.3675, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 1.079000
Epoch 280
Validation binary_cross_entropy = 1.128458
Epoch 281
Validation binary_cross_entropy = 1.166914
Epoch 282
Validation binary_cross_entropy = 1.153983
Epoch 283
Validation binary_cross_entropy = 1.138781
Epoch 284
Loss = 5.3274e-02, PNorm = 67.9481, GNorm = 1.3720, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 1.133167
Epoch 285
Validation binary_cross_entropy = 1.154596
Epoch 286
Validation binary_cross_entropy = 1.093543
Epoch 287
Validation binary_cross_entropy = 1.023464
Epoch 288
Validation binary_cross_entropy = 1.006403
Epoch 289
Loss = 2.4342e-02, PNorm = 67.9829, GNorm = 1.6654, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 1.008850
Epoch 290
Validation binary_cross_entropy = 1.018823
Epoch 291
Validation binary_cross_entropy = 1.037742
Epoch 292
Validation binary_cross_entropy = 1.036773
Epoch 293
Validation binary_cross_entropy = 1.043579
Epoch 294
Loss = 7.0212e-02, PNorm = 68.0159, GNorm = 1.0527, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 1.067984
Epoch 295
Validation binary_cross_entropy = 1.085332
Epoch 296
Validation binary_cross_entropy = 1.102294
Epoch 297
Validation binary_cross_entropy = 1.114647
Epoch 298
Validation binary_cross_entropy = 1.204197
Epoch 299
Loss = 5.7316e-02, PNorm = 68.0507, GNorm = 3.8868, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 1.227961
Model 0 best validation binary_cross_entropy = 0.287546 on epoch 0
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.225327
Ensemble test binary_cross_entropy = 0.225327
Fold 9
Splitting data with seed 9
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.4, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=800, bias=False)
        (W_h): Linear(in_features=800, out_features=800, bias=False)
        (W_o): Linear(in_features=933, out_features=800, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.4, inplace=False)
    (1): Linear(in_features=3100, out_features=800, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.4, inplace=False)
    (4): Linear(in_features=800, out_features=800, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.4, inplace=False)
    (7): Linear(in_features=800, out_features=1, bias=True)
  )
)
Number of parameters = 4,627,201
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 0.288900
Epoch 1
Validation binary_cross_entropy = 0.296113
Epoch 2
Validation binary_cross_entropy = 0.627527
Epoch 3
Validation binary_cross_entropy = 0.325645
Epoch 4
Loss = 8.3044e-01, PNorm = 63.1499, GNorm = 6.1946, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 0.298872
Epoch 5
Validation binary_cross_entropy = 0.414138
Epoch 6
Validation binary_cross_entropy = 0.989933
Epoch 7
Validation binary_cross_entropy = 0.306474
Epoch 8
Validation binary_cross_entropy = 0.312818
Epoch 9
Loss = 6.8364e-01, PNorm = 63.2825, GNorm = 5.0200, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 0.934482
Epoch 10
Validation binary_cross_entropy = 0.547546
Epoch 11
Validation binary_cross_entropy = 0.309192
Epoch 12
Validation binary_cross_entropy = 0.598920
Epoch 13
Validation binary_cross_entropy = 0.757241
Epoch 14
Loss = 7.5174e-01, PNorm = 63.4329, GNorm = 4.7699, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 0.528832
Epoch 15
Validation binary_cross_entropy = 0.518389
Epoch 16
Validation binary_cross_entropy = 0.456228
Epoch 17
Validation binary_cross_entropy = 0.651107
Epoch 18
Validation binary_cross_entropy = 0.637769
Epoch 19
Loss = 7.0490e-01, PNorm = 63.5508, GNorm = 4.1631, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 0.411958
Epoch 20
Validation binary_cross_entropy = 0.379109
Epoch 21
Validation binary_cross_entropy = 0.663031
Epoch 22
Validation binary_cross_entropy = 0.654711
Epoch 23
Validation binary_cross_entropy = 0.513110
Epoch 24
Loss = 6.4934e-01, PNorm = 63.6422, GNorm = 4.8818, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 0.425396
Epoch 25
Validation binary_cross_entropy = 0.566951
Epoch 26
Validation binary_cross_entropy = 0.563345
Epoch 27
Validation binary_cross_entropy = 0.511907
Epoch 28
Validation binary_cross_entropy = 0.524909
Epoch 29
Loss = 5.9170e-01, PNorm = 63.7176, GNorm = 3.1235, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 0.557497
Epoch 30
Validation binary_cross_entropy = 0.508854
Epoch 31
Validation binary_cross_entropy = 0.505205
Epoch 32
Validation binary_cross_entropy = 0.682773
Epoch 33
Validation binary_cross_entropy = 0.551893
Epoch 34
Loss = 5.6338e-01, PNorm = 63.7647, GNorm = 2.2943, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 0.402073
Epoch 35
Validation binary_cross_entropy = 0.421800
Epoch 36
Validation binary_cross_entropy = 0.516861
Epoch 37
Validation binary_cross_entropy = 0.551832
Epoch 38
Validation binary_cross_entropy = 0.451983
Epoch 39
Loss = 7.0629e-01, PNorm = 63.8176, GNorm = 2.3840, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 0.445434
Epoch 40
Validation binary_cross_entropy = 0.500389
Epoch 41
Validation binary_cross_entropy = 0.567812
Epoch 42
Validation binary_cross_entropy = 0.546756
Epoch 43
Validation binary_cross_entropy = 0.509498
Epoch 44
Loss = 6.4506e-01, PNorm = 63.8690, GNorm = 2.4382, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 0.499064
Epoch 45
Validation binary_cross_entropy = 0.532511
Epoch 46
Validation binary_cross_entropy = 0.562847
Epoch 47
Validation binary_cross_entropy = 0.552566
Epoch 48
Validation binary_cross_entropy = 0.488790
Epoch 49
Loss = 6.0301e-01, PNorm = 63.9332, GNorm = 2.8309, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 0.509063
Epoch 50
Validation binary_cross_entropy = 0.425102
Epoch 51
Validation binary_cross_entropy = 0.462721
Epoch 52
Validation binary_cross_entropy = 0.551021
Epoch 53
Validation binary_cross_entropy = 0.589781
Epoch 54
Loss = 6.1229e-01, PNorm = 63.9844, GNorm = 2.2209, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 0.512728
Epoch 55
Validation binary_cross_entropy = 0.450623
Epoch 56
Validation binary_cross_entropy = 0.463310
Epoch 57
Validation binary_cross_entropy = 0.459335
Epoch 58
Validation binary_cross_entropy = 0.474010
Epoch 59
Loss = 5.5366e-01, PNorm = 64.0474, GNorm = 1.6926, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 0.503369
Epoch 60
Validation binary_cross_entropy = 0.505439
Epoch 61
Validation binary_cross_entropy = 0.500602
Epoch 62
Validation binary_cross_entropy = 0.497277
Epoch 63
Validation binary_cross_entropy = 0.481739
Epoch 64
Loss = 6.1077e-01, PNorm = 64.1116, GNorm = 1.5692, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 0.490339
Epoch 65
Validation binary_cross_entropy = 0.506147
Epoch 66
Validation binary_cross_entropy = 0.568908
Epoch 67
Validation binary_cross_entropy = 0.517063
Epoch 68
Validation binary_cross_entropy = 0.460587
Epoch 69
Loss = 5.2219e-01, PNorm = 64.1851, GNorm = 1.6659, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 0.461268
Epoch 70
Validation binary_cross_entropy = 0.541916
Epoch 71
Validation binary_cross_entropy = 0.515901
Epoch 72
Validation binary_cross_entropy = 0.422997
Epoch 73
Validation binary_cross_entropy = 0.424317
Epoch 74
Loss = 4.9852e-01, PNorm = 64.2593, GNorm = 2.2279, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 0.695956
Epoch 75
Validation binary_cross_entropy = 0.474141
Epoch 76
Validation binary_cross_entropy = 0.380577
Epoch 77
Validation binary_cross_entropy = 0.509671
Epoch 78
Validation binary_cross_entropy = 0.649239
Epoch 79
Loss = 5.6326e-01, PNorm = 64.3305, GNorm = 2.1418, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 0.601090
Epoch 80
Validation binary_cross_entropy = 0.461573
Epoch 81
Validation binary_cross_entropy = 0.446582
Epoch 82
Validation binary_cross_entropy = 0.552948
Epoch 83
Validation binary_cross_entropy = 0.568176
Epoch 84
Loss = 5.2070e-01, PNorm = 64.4223, GNorm = 1.5745, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 0.425082
Epoch 85
Validation binary_cross_entropy = 0.661683
Epoch 86
Validation binary_cross_entropy = 0.649528
Epoch 87
Validation binary_cross_entropy = 0.370485
Epoch 88
Validation binary_cross_entropy = 0.467967
Epoch 89
Loss = 5.2102e-01, PNorm = 64.5004, GNorm = 1.8889, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 0.570789
Epoch 90
Validation binary_cross_entropy = 0.531688
Epoch 91
Validation binary_cross_entropy = 0.448440
Epoch 92
Validation binary_cross_entropy = 0.421496
Epoch 93
Validation binary_cross_entropy = 0.372979
Epoch 94
Loss = 4.9588e-01, PNorm = 64.5905, GNorm = 1.6387, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 0.417093
Epoch 95
Validation binary_cross_entropy = 0.459382
Epoch 96
Validation binary_cross_entropy = 0.401655
Epoch 97
Validation binary_cross_entropy = 0.400023
Epoch 98
Validation binary_cross_entropy = 0.435690
Epoch 99
Loss = 6.7552e-01, PNorm = 64.6970, GNorm = 6.3244, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.360922
Epoch 100
Validation binary_cross_entropy = 0.453231
Epoch 101
Validation binary_cross_entropy = 0.692516
Epoch 102
Validation binary_cross_entropy = 0.513603
Epoch 103
Validation binary_cross_entropy = 0.398976
Epoch 104
Loss = 4.7763e-01, PNorm = 64.7862, GNorm = 1.3288, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.360552
Epoch 105
Validation binary_cross_entropy = 0.414690
Epoch 106
Validation binary_cross_entropy = 0.504618
Epoch 107
Validation binary_cross_entropy = 0.561164
Epoch 108
Validation binary_cross_entropy = 0.480846
Epoch 109
Loss = 4.2861e-01, PNorm = 64.8959, GNorm = 2.3231, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.404064
Epoch 110
Validation binary_cross_entropy = 0.435686
Epoch 111
Validation binary_cross_entropy = 0.449121
Epoch 112
Validation binary_cross_entropy = 0.405311
Epoch 113
Validation binary_cross_entropy = 0.410347
Epoch 114
Loss = 5.1305e-01, PNorm = 64.9679, GNorm = 1.6515, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.447559
Epoch 115
Validation binary_cross_entropy = 0.422570
Epoch 116
Validation binary_cross_entropy = 0.444012
Epoch 117
Validation binary_cross_entropy = 0.525262
Epoch 118
Validation binary_cross_entropy = 0.411156
Epoch 119
Loss = 4.0218e-01, PNorm = 65.0634, GNorm = 1.5317, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.405639
Epoch 120
Validation binary_cross_entropy = 0.410765
Epoch 121
Validation binary_cross_entropy = 0.427659
Epoch 122
Validation binary_cross_entropy = 0.405211
Epoch 123
Validation binary_cross_entropy = 0.442911
Epoch 124
Loss = 3.0890e-01, PNorm = 65.1671, GNorm = 1.6124, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.394777
Epoch 125
Validation binary_cross_entropy = 0.590893
Epoch 126
Validation binary_cross_entropy = 0.406036
Epoch 127
Validation binary_cross_entropy = 0.428446
Epoch 128
Validation binary_cross_entropy = 0.475835
Epoch 129
Loss = 2.7375e-01, PNorm = 65.2914, GNorm = 1.6055, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.474203
Epoch 130
Validation binary_cross_entropy = 0.435007
Epoch 131
Validation binary_cross_entropy = 0.577607
Epoch 132
Validation binary_cross_entropy = 0.355320
Epoch 133
Validation binary_cross_entropy = 0.376572
Epoch 134
Loss = 3.5452e-01, PNorm = 65.4069, GNorm = 3.2060, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.556744
Epoch 135
Validation binary_cross_entropy = 0.391409
Epoch 136
Validation binary_cross_entropy = 0.340950
Epoch 137
Validation binary_cross_entropy = 0.413061
Epoch 138
Validation binary_cross_entropy = 0.511444
Epoch 139
Loss = 2.1878e-01, PNorm = 65.5293, GNorm = 1.8538, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.610915
Epoch 140
Validation binary_cross_entropy = 0.571178
Epoch 141
Validation binary_cross_entropy = 0.543512
Epoch 142
Validation binary_cross_entropy = 0.520974
Epoch 143
Validation binary_cross_entropy = 0.574298
Epoch 144
Loss = 3.0774e-01, PNorm = 65.6556, GNorm = 2.9348, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.389428
Epoch 145
Validation binary_cross_entropy = 0.369535
Epoch 146
Validation binary_cross_entropy = 0.412468
Epoch 147
Validation binary_cross_entropy = 0.516225
Epoch 148
Validation binary_cross_entropy = 0.455079
Epoch 149
Loss = 2.4803e-01, PNorm = 65.7841, GNorm = 2.4804, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.451709
Epoch 150
Validation binary_cross_entropy = 0.481297
Epoch 151
Validation binary_cross_entropy = 0.581413
Epoch 152
Validation binary_cross_entropy = 0.496244
Epoch 153
Validation binary_cross_entropy = 0.536737
Epoch 154
Loss = 2.9935e-01, PNorm = 65.9096, GNorm = 2.0373, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.504525
Epoch 155
Validation binary_cross_entropy = 0.532952
Epoch 156
Validation binary_cross_entropy = 0.568003
Epoch 157
Validation binary_cross_entropy = 0.622010
Epoch 158
Validation binary_cross_entropy = 0.691760
Epoch 159
Loss = 2.1128e-01, PNorm = 66.0280, GNorm = 2.7696, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.680182
Epoch 160
Validation binary_cross_entropy = 0.674138
Epoch 161
Validation binary_cross_entropy = 0.728066
Epoch 162
Validation binary_cross_entropy = 0.690325
Epoch 163
Validation binary_cross_entropy = 0.691302
Epoch 164
Loss = 2.6650e-01, PNorm = 66.1355, GNorm = 3.8853, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.615440
Epoch 165
Validation binary_cross_entropy = 0.651972
Epoch 166
Validation binary_cross_entropy = 0.674294
Epoch 167
Validation binary_cross_entropy = 0.675057
Epoch 168
Validation binary_cross_entropy = 0.655449
Epoch 169
Loss = 2.4711e-01, PNorm = 66.2601, GNorm = 3.1258, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.625582
Epoch 170
Validation binary_cross_entropy = 0.685914
Epoch 171
Validation binary_cross_entropy = 0.578064
Epoch 172
Validation binary_cross_entropy = 0.594077
Epoch 173
Validation binary_cross_entropy = 0.608796
Epoch 174
Loss = 2.2821e-01, PNorm = 66.3676, GNorm = 1.6336, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 0.612264
Epoch 175
Validation binary_cross_entropy = 0.622833
Epoch 176
Validation binary_cross_entropy = 0.655566
Epoch 177
Validation binary_cross_entropy = 0.730178
Epoch 178
Validation binary_cross_entropy = 0.675535
Epoch 179
Loss = 2.1640e-01, PNorm = 66.4821, GNorm = 3.0202, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 0.635750
Epoch 180
Validation binary_cross_entropy = 0.631667
Epoch 181
Validation binary_cross_entropy = 0.655540
Epoch 182
Validation binary_cross_entropy = 0.664697
Epoch 183
Validation binary_cross_entropy = 0.668432
Epoch 184
Loss = 1.1927e-01, PNorm = 66.5751, GNorm = 1.7089, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 0.679258
Epoch 185
Validation binary_cross_entropy = 0.718565
Epoch 186
Validation binary_cross_entropy = 0.710256
Epoch 187
Validation binary_cross_entropy = 0.765935
Epoch 188
Validation binary_cross_entropy = 0.761811
Epoch 189
Loss = 1.4232e-01, PNorm = 66.6724, GNorm = 1.1739, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 0.659185
Epoch 190
Validation binary_cross_entropy = 0.652364
Epoch 191
Validation binary_cross_entropy = 0.806517
Epoch 192
Validation binary_cross_entropy = 0.730548
Epoch 193
Validation binary_cross_entropy = 0.758978
Epoch 194
Loss = 1.6751e-01, PNorm = 66.7763, GNorm = 3.7780, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 0.785842
Epoch 195
Validation binary_cross_entropy = 0.761353
Epoch 196
Validation binary_cross_entropy = 0.827404
Epoch 197
Validation binary_cross_entropy = 0.715253
Epoch 198
Validation binary_cross_entropy = 0.685623
Epoch 199
Loss = 1.3429e-01, PNorm = 66.8715, GNorm = 2.4346, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 0.727729
Epoch 200
Validation binary_cross_entropy = 0.787337
Epoch 201
Validation binary_cross_entropy = 0.888449
Epoch 202
Validation binary_cross_entropy = 0.862079
Epoch 203
Validation binary_cross_entropy = 0.858547
Epoch 204
Loss = 2.3100e-01, PNorm = 66.9509, GNorm = 1.2725, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 0.952700
Epoch 205
Validation binary_cross_entropy = 0.940463
Epoch 206
Validation binary_cross_entropy = 0.833851
Epoch 207
Validation binary_cross_entropy = 0.856566
Epoch 208
Validation binary_cross_entropy = 0.805459
Epoch 209
Loss = 4.8214e-02, PNorm = 67.0231, GNorm = 0.6904, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 0.947593
Epoch 210
Validation binary_cross_entropy = 0.856513
Epoch 211
Validation binary_cross_entropy = 0.780611
Epoch 212
Validation binary_cross_entropy = 0.789957
Epoch 213
Validation binary_cross_entropy = 0.844410
Epoch 214
Loss = 9.6699e-02, PNorm = 67.0992, GNorm = 1.8094, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 0.959258
Epoch 215
Validation binary_cross_entropy = 0.997409
Epoch 216
Validation binary_cross_entropy = 1.025235
Epoch 217
Validation binary_cross_entropy = 1.052710
Epoch 218
Validation binary_cross_entropy = 1.092445
Epoch 219
Loss = 7.7272e-02, PNorm = 67.1685, GNorm = 0.8569, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 1.139068
Epoch 220
Validation binary_cross_entropy = 1.094252
Epoch 221
Validation binary_cross_entropy = 1.032983
Epoch 222
Validation binary_cross_entropy = 1.005158
Epoch 223
Validation binary_cross_entropy = 0.996140
Epoch 224
Loss = 1.1100e-01, PNorm = 67.2390, GNorm = 0.7395, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 1.004528
Epoch 225
Validation binary_cross_entropy = 1.010194
Epoch 226
Validation binary_cross_entropy = 1.001788
Epoch 227
Validation binary_cross_entropy = 1.081595
Epoch 228
Validation binary_cross_entropy = 1.133931
Epoch 229
Loss = 1.4656e-01, PNorm = 67.3158, GNorm = 0.9273, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 0.923799
Epoch 230
Validation binary_cross_entropy = 0.903259
Epoch 231
Validation binary_cross_entropy = 0.943619
Epoch 232
Validation binary_cross_entropy = 0.983836
Epoch 233
Validation binary_cross_entropy = 0.760726
Epoch 234
Loss = 1.0202e-01, PNorm = 67.3863, GNorm = 2.3109, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 0.782940
Epoch 235
Validation binary_cross_entropy = 0.754580
Epoch 236
Validation binary_cross_entropy = 0.827625
Epoch 237
Validation binary_cross_entropy = 0.960367
Epoch 238
Validation binary_cross_entropy = 0.967375
Epoch 239
Loss = 9.3937e-02, PNorm = 67.4737, GNorm = 1.4101, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 0.976392
Epoch 240
Validation binary_cross_entropy = 1.004143
Epoch 241
Validation binary_cross_entropy = 1.125548
Epoch 242
Validation binary_cross_entropy = 1.304341
Epoch 243
Validation binary_cross_entropy = 1.015541
Epoch 244
Loss = 1.0096e-01, PNorm = 67.5400, GNorm = 0.7349, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 0.929505
Epoch 245
Validation binary_cross_entropy = 0.912798
Epoch 246
Validation binary_cross_entropy = 0.920925
Epoch 247
Validation binary_cross_entropy = 1.021226
Epoch 248
Validation binary_cross_entropy = 0.999889
Epoch 249
Loss = 5.1999e-02, PNorm = 67.6054, GNorm = 1.0720, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 0.903183
Epoch 250
Validation binary_cross_entropy = 0.923696
Epoch 251
Validation binary_cross_entropy = 1.000984
Epoch 252
Validation binary_cross_entropy = 0.945743
Epoch 253
Validation binary_cross_entropy = 0.879110
Epoch 254
Loss = 7.3290e-02, PNorm = 67.6626, GNorm = 0.6395, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 0.863368
Epoch 255
Validation binary_cross_entropy = 0.880684
Epoch 256
Validation binary_cross_entropy = 0.906490
Epoch 257
Validation binary_cross_entropy = 0.933370
Epoch 258
Validation binary_cross_entropy = 0.962841
Epoch 259
Loss = 5.8618e-02, PNorm = 67.7211, GNorm = 0.3700, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 1.029311
Epoch 260
Validation binary_cross_entropy = 1.102662
Epoch 261
Validation binary_cross_entropy = 1.198411
Epoch 262
Validation binary_cross_entropy = 1.314830
Epoch 263
Validation binary_cross_entropy = 1.335524
Epoch 264
Loss = 3.7244e-02, PNorm = 67.7764, GNorm = 0.2778, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 1.234284
Epoch 265
Validation binary_cross_entropy = 1.247617
Epoch 266
Validation binary_cross_entropy = 1.204413
Epoch 267
Validation binary_cross_entropy = 1.306379
Epoch 268
Validation binary_cross_entropy = 1.207617
Epoch 269
Loss = 5.7970e-02, PNorm = 67.8184, GNorm = 1.6239, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 1.115563
Epoch 270
Validation binary_cross_entropy = 1.126639
Epoch 271
Validation binary_cross_entropy = 1.175565
Epoch 272
Validation binary_cross_entropy = 1.278191
Epoch 273
Validation binary_cross_entropy = 1.281467
Epoch 274
Loss = 6.7799e-02, PNorm = 67.8727, GNorm = 1.2204, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 1.275515
Epoch 275
Validation binary_cross_entropy = 1.318951
Epoch 276
Validation binary_cross_entropy = 1.365902
Epoch 277
Validation binary_cross_entropy = 1.440852
Epoch 278
Validation binary_cross_entropy = 1.441641
Epoch 279
Loss = 5.7664e-02, PNorm = 67.9232, GNorm = 1.5768, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 1.499216
Epoch 280
Validation binary_cross_entropy = 1.490388
Epoch 281
Validation binary_cross_entropy = 1.520262
Epoch 282
Validation binary_cross_entropy = 1.535105
Epoch 283
Validation binary_cross_entropy = 1.727079
Epoch 284
Loss = 2.7786e-02, PNorm = 67.9711, GNorm = 2.1677, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 1.763739
Epoch 285
Validation binary_cross_entropy = 1.645660
Epoch 286
Validation binary_cross_entropy = 1.650072
Epoch 287
Validation binary_cross_entropy = 1.713339
Epoch 288
Validation binary_cross_entropy = 1.795808
Epoch 289
Loss = 1.5754e-02, PNorm = 68.0131, GNorm = 2.4261, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 1.862291
Epoch 290
Validation binary_cross_entropy = 1.805245
Epoch 291
Validation binary_cross_entropy = 1.762914
Epoch 292
Validation binary_cross_entropy = 1.975118
Epoch 293
Validation binary_cross_entropy = 2.003293
Epoch 294
Loss = 7.3219e-02, PNorm = 68.0479, GNorm = 3.3368, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 1.808892
Epoch 295
Validation binary_cross_entropy = 1.630189
Epoch 296
Validation binary_cross_entropy = 1.545518
Epoch 297
Validation binary_cross_entropy = 1.543763
Epoch 298
Validation binary_cross_entropy = 1.596632
Epoch 299
Loss = 9.9779e-03, PNorm = 68.0834, GNorm = 0.4457, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 1.636361
Model 0 best validation binary_cross_entropy = 0.288900 on epoch 0
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.222776
Ensemble test binary_cross_entropy = 0.222776
10-fold cross validation
	Seed 0 ==> test binary_cross_entropy = 0.242493
	Seed 1 ==> test binary_cross_entropy = 0.198337
	Seed 2 ==> test binary_cross_entropy = 0.205699
	Seed 3 ==> test binary_cross_entropy = 0.244754
	Seed 4 ==> test binary_cross_entropy = 0.193636
	Seed 5 ==> test binary_cross_entropy = 0.211122
	Seed 6 ==> test binary_cross_entropy = 0.227363
	Seed 7 ==> test binary_cross_entropy = 0.223729
	Seed 8 ==> test binary_cross_entropy = 0.225327
	Seed 9 ==> test binary_cross_entropy = 0.222776
Overall test binary_cross_entropy = 0.219524 +/- 0.016335
Elapsed time = 0:05:19
Command line
python /home/saxelrod/Repo/projects/chemprop/chemprop/train.py --config_path /home/saxelrod/chemprop_cov_2/models/from_cp_feats_mpnn_from_binary_cross_entropy/config.json --data_path /home/saxelrod/chemprop_cov_2/scaffold_split/train_full.csv --dataset_type classification
Args
{'activation': 'ReLU',
 'aggregation': 'mean',
 'aggregation_norm': 100,
 'atom_descriptors': None,
 'atom_descriptors_path': None,
 'atom_descriptors_size': 0,
 'atom_features_size': 0,
 'atom_messages': False,
 'batch_size': 50,
 'bias': False,
 'cache_cutoff': 10000,
 'checkpoint_dir': None,
 'checkpoint_path': None,
 'checkpoint_paths': None,
 'class_balance': True,
 'config_path': '/home/saxelrod/chemprop_cov_2/models/from_cp_feats_mpnn_from_binary_cross_entropy/config.json',
 'crossval_index_dir': None,
 'crossval_index_file': None,
 'crossval_index_sets': None,
 'cuda': True,
 'data_path': '/home/saxelrod/chemprop_cov_2/scaffold_split/train_full.csv',
 'dataset_type': 'classification',
 'depth': 6,
 'device': device(type='cuda', index=1),
 'dropout': 0.4,
 'ensemble_size': 1,
 'epochs': 300,
 'extra_metrics': [],
 'features_generator': None,
 'features_only': False,
 'features_path': ['/home/saxelrod/chemprop_cov_2/features/from_cp/train_binary_cross_entropy.npz'],
 'features_scaling': False,
 'features_size': None,
 'ffn_hidden_size': 800,
 'ffn_num_layers': 3,
 'final_lr': 0.0001,
 'folds_file': None,
 'gpu': 1,
 'grad_clip': None,
 'hidden_size': 800,
 'ignore_columns': None,
 'init_lr': 0.0001,
 'log_frequency': 10,
 'max_data_size': None,
 'max_lr': 0.001,
 'metric': 'binary_cross_entropy',
 'metrics': ['binary_cross_entropy'],
 'minimize_score': True,
 'mpn_shared': False,
 'multiclass_num_classes': 3,
 'no_cache_mol': False,
 'no_cuda': False,
 'no_features_scaling': True,
 'num_folds': 10,
 'num_lrs': 1,
 'num_tasks': 1,
 'num_workers': 8,
 'number_of_molecules': 1,
 'pytorch_seed': 0,
 'quiet': True,
 'save_dir': '/home/saxelrod/chemprop_cov_2/models/from_cp_feats_mpnn_from_binary_cross_entropy',
 'save_preds': False,
 'save_smiles_splits': False,
 'seed': 0,
 'separate_test_features_path': ['/home/saxelrod/chemprop_cov_2/features/from_cp/test_binary_cross_entropy.npz'],
 'separate_test_path': '/home/saxelrod/chemprop_cov_2/scaffold_split/test_full.csv',
 'separate_val_features_path': ['/home/saxelrod/chemprop_cov_2/features/from_cp/val_binary_cross_entropy.npz'],
 'separate_val_path': '/home/saxelrod/chemprop_cov_2/scaffold_split/val_full.csv',
 'show_individual_scores': False,
 'smiles_columns': [None],
 'split_sizes': (0.8, 0.1, 0.1),
 'split_type': 'random',
 'target_columns': None,
 'task_names': ['sars_cov_two_cl_protease_active'],
 'test': False,
 'test_fold_index': None,
 'train_data_size': None,
 'undirected': False,
 'use_input_features': True,
 'val_fold_index': None,
 'warmup_epochs': 2.0}
Loading data
Number of tasks = 1
Fold 0
Splitting data with seed 0
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.4, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=800, bias=False)
        (W_h): Linear(in_features=800, out_features=800, bias=False)
        (W_o): Linear(in_features=933, out_features=800, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.4, inplace=False)
    (1): Linear(in_features=3100, out_features=800, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.4, inplace=False)
    (4): Linear(in_features=800, out_features=800, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.4, inplace=False)
    (7): Linear(in_features=800, out_features=1, bias=True)
  )
)
Number of parameters = 4,627,201
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 0.291513
Epoch 1
Validation binary_cross_entropy = 0.300563
Epoch 2
Validation binary_cross_entropy = 0.549332
Epoch 3
Validation binary_cross_entropy = 0.331805
Epoch 4
Loss = 8.2302e-01, PNorm = 63.1518, GNorm = 7.3082, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 0.310001
Epoch 5
Validation binary_cross_entropy = 0.413272
Epoch 6
Validation binary_cross_entropy = 0.632496
Epoch 7
Validation binary_cross_entropy = 0.291358
Epoch 8
Validation binary_cross_entropy = 0.298074
Epoch 9
Loss = 7.6393e-01, PNorm = 63.2881, GNorm = 5.3321, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 0.638309
Epoch 10
Validation binary_cross_entropy = 0.450271
Epoch 11
Validation binary_cross_entropy = 0.313449
Epoch 12
Validation binary_cross_entropy = 0.491421
Epoch 13
Validation binary_cross_entropy = 0.476504
Epoch 14
Loss = 8.1098e-01, PNorm = 63.4439, GNorm = 4.9537, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 0.327053
Epoch 15
Validation binary_cross_entropy = 0.434943
Epoch 16
Validation binary_cross_entropy = 0.539258
Epoch 17
Validation binary_cross_entropy = 0.415537
Epoch 18
Validation binary_cross_entropy = 0.527824
Epoch 19
Loss = 6.9978e-01, PNorm = 63.5571, GNorm = 4.2303, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 0.408644
Epoch 20
Validation binary_cross_entropy = 0.369876
Epoch 21
Validation binary_cross_entropy = 0.589452
Epoch 22
Validation binary_cross_entropy = 0.481779
Epoch 23
Validation binary_cross_entropy = 0.485017
Epoch 24
Loss = 6.6369e-01, PNorm = 63.6398, GNorm = 2.8893, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 0.437743
Epoch 25
Validation binary_cross_entropy = 0.585645
Epoch 26
Validation binary_cross_entropy = 0.537080
Epoch 27
Validation binary_cross_entropy = 0.552320
Epoch 28
Validation binary_cross_entropy = 0.499481
Epoch 29
Loss = 5.9060e-01, PNorm = 63.7167, GNorm = 2.8534, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 0.480392
Epoch 30
Validation binary_cross_entropy = 0.443153
Epoch 31
Validation binary_cross_entropy = 0.530520
Epoch 32
Validation binary_cross_entropy = 0.685076
Epoch 33
Validation binary_cross_entropy = 0.514649
Epoch 34
Loss = 7.3160e-01, PNorm = 63.8002, GNorm = 5.0103, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 0.427233
Epoch 35
Validation binary_cross_entropy = 0.507239
Epoch 36
Validation binary_cross_entropy = 0.536830
Epoch 37
Validation binary_cross_entropy = 0.542240
Epoch 38
Validation binary_cross_entropy = 0.477961
Epoch 39
Loss = 5.9979e-01, PNorm = 63.8617, GNorm = 2.2587, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 0.490366
Epoch 40
Validation binary_cross_entropy = 0.501424
Epoch 41
Validation binary_cross_entropy = 0.493479
Epoch 42
Validation binary_cross_entropy = 0.478281
Epoch 43
Validation binary_cross_entropy = 0.497873
Epoch 44
Loss = 6.4639e-01, PNorm = 63.9125, GNorm = 2.9476, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 0.497061
Epoch 45
Validation binary_cross_entropy = 0.504444
Epoch 46
Validation binary_cross_entropy = 0.524794
Epoch 47
Validation binary_cross_entropy = 0.565214
Epoch 48
Validation binary_cross_entropy = 0.571733
Epoch 49
Loss = 5.7608e-01, PNorm = 63.9690, GNorm = 2.0715, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 0.566454
Epoch 50
Validation binary_cross_entropy = 0.489663
Epoch 51
Validation binary_cross_entropy = 0.466363
Epoch 52
Validation binary_cross_entropy = 0.527071
Epoch 53
Validation binary_cross_entropy = 0.585976
Epoch 54
Loss = 5.8842e-01, PNorm = 64.0205, GNorm = 1.6390, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 0.606649
Epoch 55
Validation binary_cross_entropy = 0.569462
Epoch 56
Validation binary_cross_entropy = 0.489244
Epoch 57
Validation binary_cross_entropy = 0.450669
Epoch 58
Validation binary_cross_entropy = 0.498745
Epoch 59
Loss = 5.4210e-01, PNorm = 64.0860, GNorm = 2.1165, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 0.594303
Epoch 60
Validation binary_cross_entropy = 0.565601
Epoch 61
Validation binary_cross_entropy = 0.526246
Epoch 62
Validation binary_cross_entropy = 0.553257
Epoch 63
Validation binary_cross_entropy = 0.365143
Epoch 64
Loss = 5.6708e-01, PNorm = 64.1581, GNorm = 3.8693, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 0.424702
Epoch 65
Validation binary_cross_entropy = 0.734818
Epoch 66
Validation binary_cross_entropy = 0.593481
Epoch 67
Validation binary_cross_entropy = 0.479614
Epoch 68
Validation binary_cross_entropy = 0.436375
Epoch 69
Loss = 4.8166e-01, PNorm = 64.2429, GNorm = 1.3508, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 0.496648
Epoch 70
Validation binary_cross_entropy = 0.448676
Epoch 71
Validation binary_cross_entropy = 0.368955
Epoch 72
Validation binary_cross_entropy = 0.416038
Epoch 73
Validation binary_cross_entropy = 0.405116
Epoch 74
Loss = 4.6212e-01, PNorm = 64.3381, GNorm = 1.7301, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 0.385286
Epoch 75
Validation binary_cross_entropy = 0.317430
Epoch 76
Validation binary_cross_entropy = 0.640763
Epoch 77
Validation binary_cross_entropy = 0.456983
Epoch 78
Validation binary_cross_entropy = 0.391593
Epoch 79
Loss = 5.0234e-01, PNorm = 64.4190, GNorm = 2.3067, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 0.424095
Epoch 80
Validation binary_cross_entropy = 0.564847
Epoch 81
Validation binary_cross_entropy = 0.729193
Epoch 82
Validation binary_cross_entropy = 0.543196
Epoch 83
Validation binary_cross_entropy = 0.400939
Epoch 84
Loss = 5.0515e-01, PNorm = 64.5331, GNorm = 2.1307, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 0.419389
Epoch 85
Validation binary_cross_entropy = 0.588215
Epoch 86
Validation binary_cross_entropy = 0.423592
Epoch 87
Validation binary_cross_entropy = 0.382869
Epoch 88
Validation binary_cross_entropy = 0.428927
Epoch 89
Loss = 4.7803e-01, PNorm = 64.6275, GNorm = 1.5439, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 0.550175
Epoch 90
Validation binary_cross_entropy = 0.486576
Epoch 91
Validation binary_cross_entropy = 0.395095
Epoch 92
Validation binary_cross_entropy = 0.368515
Epoch 93
Validation binary_cross_entropy = 0.364157
Epoch 94
Loss = 3.8211e-01, PNorm = 64.7266, GNorm = 1.6379, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 0.492058
Epoch 95
Validation binary_cross_entropy = 0.335302
Epoch 96
Validation binary_cross_entropy = 0.334527
Epoch 97
Validation binary_cross_entropy = 0.423339
Epoch 98
Validation binary_cross_entropy = 0.435534
Epoch 99
Loss = 4.0832e-01, PNorm = 64.8207, GNorm = 1.9100, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.332675
Epoch 100
Validation binary_cross_entropy = 0.340582
Epoch 101
Validation binary_cross_entropy = 0.458380
Epoch 102
Validation binary_cross_entropy = 0.325717
Epoch 103
Validation binary_cross_entropy = 0.721613
Epoch 104
Loss = 4.9358e-01, PNorm = 64.9307, GNorm = 3.6929, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.539421
Epoch 105
Validation binary_cross_entropy = 0.292342
Epoch 106
Validation binary_cross_entropy = 0.315638
Epoch 107
Validation binary_cross_entropy = 0.412613
Epoch 108
Validation binary_cross_entropy = 0.486160
Epoch 109
Loss = 4.4880e-01, PNorm = 65.0490, GNorm = 1.6870, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.335090
Epoch 110
Validation binary_cross_entropy = 0.294233
Epoch 111
Validation binary_cross_entropy = 0.305402
Epoch 112
Validation binary_cross_entropy = 0.400336
Epoch 113
Validation binary_cross_entropy = 0.464611
Epoch 114
Loss = 3.8443e-01, PNorm = 65.1662, GNorm = 1.5615, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.391361
Epoch 115
Validation binary_cross_entropy = 0.396203
Epoch 116
Validation binary_cross_entropy = 0.513497
Epoch 117
Validation binary_cross_entropy = 0.418178
Epoch 118
Validation binary_cross_entropy = 0.388487
Epoch 119
Loss = 2.5380e-01, PNorm = 65.2913, GNorm = 1.3731, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.613354
Epoch 120
Validation binary_cross_entropy = 0.474538
Epoch 121
Validation binary_cross_entropy = 0.372315
Epoch 122
Validation binary_cross_entropy = 0.433803
Epoch 123
Validation binary_cross_entropy = 0.484063
Epoch 124
Loss = 3.0119e-01, PNorm = 65.4197, GNorm = 1.3540, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.401010
Epoch 125
Validation binary_cross_entropy = 0.420512
Epoch 126
Validation binary_cross_entropy = 0.483132
Epoch 127
Validation binary_cross_entropy = 0.374444
Epoch 128
Validation binary_cross_entropy = 0.332217
Epoch 129
Loss = 2.8078e-01, PNorm = 65.5430, GNorm = 1.7484, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.350676
Epoch 130
Validation binary_cross_entropy = 0.404997
Epoch 131
Validation binary_cross_entropy = 0.444125
Epoch 132
Validation binary_cross_entropy = 0.494462
Epoch 133
Validation binary_cross_entropy = 0.512761
Epoch 134
Loss = 3.1399e-01, PNorm = 65.6717, GNorm = 3.2445, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.539115
Epoch 135
Validation binary_cross_entropy = 0.498464
Epoch 136
Validation binary_cross_entropy = 0.431225
Epoch 137
Validation binary_cross_entropy = 0.433785
Epoch 138
Validation binary_cross_entropy = 0.529801
Epoch 139
Loss = 2.8763e-01, PNorm = 65.7962, GNorm = 2.5216, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.535909
Epoch 140
Validation binary_cross_entropy = 0.414471
Epoch 141
Validation binary_cross_entropy = 0.440450
Epoch 142
Validation binary_cross_entropy = 0.442936
Epoch 143
Validation binary_cross_entropy = 0.478012
Epoch 144
Loss = 2.9299e-01, PNorm = 65.9132, GNorm = 1.2119, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.443291
Epoch 145
Validation binary_cross_entropy = 0.427315
Epoch 146
Validation binary_cross_entropy = 0.428895
Epoch 147
Validation binary_cross_entropy = 0.439332
Epoch 148
Validation binary_cross_entropy = 0.470202
Epoch 149
Loss = 1.4828e-01, PNorm = 66.0316, GNorm = 1.2197, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.520475
Epoch 150
Validation binary_cross_entropy = 0.561264
Epoch 151
Validation binary_cross_entropy = 0.590115
Epoch 152
Validation binary_cross_entropy = 0.623507
Epoch 153
Validation binary_cross_entropy = 0.638113
Epoch 154
Loss = 1.7463e-01, PNorm = 66.1310, GNorm = 1.1854, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.613183
Epoch 155
Validation binary_cross_entropy = 0.607485
Epoch 156
Validation binary_cross_entropy = 0.623779
Epoch 157
Validation binary_cross_entropy = 0.591585
Epoch 158
Validation binary_cross_entropy = 0.534203
Epoch 159
Loss = 2.1993e-01, PNorm = 66.2261, GNorm = 1.2403, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.540053
Epoch 160
Validation binary_cross_entropy = 0.528178
Epoch 161
Validation binary_cross_entropy = 0.534948
Epoch 162
Validation binary_cross_entropy = 0.557417
Epoch 163
Validation binary_cross_entropy = 0.638133
Epoch 164
Loss = 2.6333e-01, PNorm = 66.3243, GNorm = 4.8232, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.555892
Epoch 165
Validation binary_cross_entropy = 0.514741
Epoch 166
Validation binary_cross_entropy = 0.492567
Epoch 167
Validation binary_cross_entropy = 0.512404
Epoch 168
Validation binary_cross_entropy = 0.558871
Epoch 169
Loss = 1.7197e-01, PNorm = 66.4210, GNorm = 2.6101, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.562796
Epoch 170
Validation binary_cross_entropy = 0.608897
Epoch 171
Validation binary_cross_entropy = 0.645343
Epoch 172
Validation binary_cross_entropy = 0.671981
Epoch 173
Validation binary_cross_entropy = 0.648177
Epoch 174
Loss = 1.5521e-01, PNorm = 66.5116, GNorm = 0.7514, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 0.655749
Epoch 175
Validation binary_cross_entropy = 0.665171
Epoch 176
Validation binary_cross_entropy = 0.688922
Epoch 177
Validation binary_cross_entropy = 0.709575
Epoch 178
Validation binary_cross_entropy = 0.716942
Epoch 179
Loss = 1.1609e-01, PNorm = 66.6013, GNorm = 0.9371, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 0.663592
Epoch 180
Validation binary_cross_entropy = 0.680874
Epoch 181
Validation binary_cross_entropy = 0.772537
Epoch 182
Validation binary_cross_entropy = 0.786908
Epoch 183
Validation binary_cross_entropy = 0.815885
Epoch 184
Loss = 9.7833e-02, PNorm = 66.6867, GNorm = 2.0965, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 0.829068
Epoch 185
Validation binary_cross_entropy = 0.782669
Epoch 186
Validation binary_cross_entropy = 0.758480
Epoch 187
Validation binary_cross_entropy = 0.709322
Epoch 188
Validation binary_cross_entropy = 0.703032
Epoch 189
Loss = 1.7394e-01, PNorm = 66.7703, GNorm = 3.2816, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 0.664423
Epoch 190
Validation binary_cross_entropy = 0.684821
Epoch 191
Validation binary_cross_entropy = 0.642785
Epoch 192
Validation binary_cross_entropy = 0.630588
Epoch 193
Validation binary_cross_entropy = 0.667887
Epoch 194
Loss = 1.6007e-01, PNorm = 66.8688, GNorm = 1.3725, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 0.760205
Epoch 195
Validation binary_cross_entropy = 0.686413
Epoch 196
Validation binary_cross_entropy = 0.668900
Epoch 197
Validation binary_cross_entropy = 0.672013
Epoch 198
Validation binary_cross_entropy = 0.723598
Epoch 199
Loss = 1.6143e-01, PNorm = 66.9533, GNorm = 1.6249, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 0.691179
Epoch 200
Validation binary_cross_entropy = 0.662706
Epoch 201
Validation binary_cross_entropy = 0.672972
Epoch 202
Validation binary_cross_entropy = 0.682454
Epoch 203
Validation binary_cross_entropy = 0.722743
Epoch 204
Loss = 1.0428e-01, PNorm = 67.0344, GNorm = 1.3031, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 0.801398
Epoch 205
Validation binary_cross_entropy = 0.842544
Epoch 206
Validation binary_cross_entropy = 0.849514
Epoch 207
Validation binary_cross_entropy = 0.890061
Epoch 208
Validation binary_cross_entropy = 0.914719
Epoch 209
Loss = 1.1801e-01, PNorm = 67.0975, GNorm = 4.6481, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 0.888746
Epoch 210
Validation binary_cross_entropy = 0.834803
Epoch 211
Validation binary_cross_entropy = 0.835002
Epoch 212
Validation binary_cross_entropy = 0.895135
Epoch 213
Validation binary_cross_entropy = 0.831179
Epoch 214
Loss = 1.0330e-01, PNorm = 67.1638, GNorm = 3.2304, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 0.716137
Epoch 215
Validation binary_cross_entropy = 0.673215
Epoch 216
Validation binary_cross_entropy = 0.685730
Epoch 217
Validation binary_cross_entropy = 0.789129
Epoch 218
Validation binary_cross_entropy = 0.789408
Epoch 219
Loss = 6.2000e-02, PNorm = 67.2472, GNorm = 0.5601, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 0.766084
Epoch 220
Validation binary_cross_entropy = 0.781463
Epoch 221
Validation binary_cross_entropy = 0.805680
Epoch 222
Validation binary_cross_entropy = 0.860668
Epoch 223
Validation binary_cross_entropy = 0.866936
Epoch 224
Loss = 8.2610e-02, PNorm = 67.3192, GNorm = 0.6525, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 0.868158
Epoch 225
Validation binary_cross_entropy = 0.854680
Epoch 226
Validation binary_cross_entropy = 0.852938
Epoch 227
Validation binary_cross_entropy = 0.875112
Epoch 228
Validation binary_cross_entropy = 0.890244
Epoch 229
Loss = 8.9978e-02, PNorm = 67.3817, GNorm = 1.5746, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 0.952227
Epoch 230
Validation binary_cross_entropy = 1.043060
Epoch 231
Validation binary_cross_entropy = 0.976974
Epoch 232
Validation binary_cross_entropy = 0.938446
Epoch 233
Validation binary_cross_entropy = 0.943160
Epoch 234
Loss = 7.4074e-02, PNorm = 67.4410, GNorm = 0.7492, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 0.959421
Epoch 235
Validation binary_cross_entropy = 0.958774
Epoch 236
Validation binary_cross_entropy = 0.969639
Epoch 237
Validation binary_cross_entropy = 1.002702
Epoch 238
Validation binary_cross_entropy = 0.959959
Epoch 239
Loss = 8.3489e-02, PNorm = 67.4944, GNorm = 2.3913, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 0.937509
Epoch 240
Validation binary_cross_entropy = 1.004860
Epoch 241
Validation binary_cross_entropy = 1.049602
Epoch 242
Validation binary_cross_entropy = 0.968815
Epoch 243
Validation binary_cross_entropy = 0.923934
Epoch 244
Loss = 6.5318e-02, PNorm = 67.5451, GNorm = 0.8162, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 0.849236
Epoch 245
Validation binary_cross_entropy = 0.839018
Epoch 246
Validation binary_cross_entropy = 0.783737
Epoch 247
Validation binary_cross_entropy = 0.737618
Epoch 248
Validation binary_cross_entropy = 0.742191
Epoch 249
Loss = 1.4853e-01, PNorm = 67.6103, GNorm = 1.6332, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 0.802002
Epoch 250
Validation binary_cross_entropy = 0.864917
Epoch 251
Validation binary_cross_entropy = 0.907121
Epoch 252
Validation binary_cross_entropy = 0.939002
Epoch 253
Validation binary_cross_entropy = 0.971051
Epoch 254
Loss = 4.7866e-02, PNorm = 67.6775, GNorm = 1.4719, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 1.015588
Epoch 255
Validation binary_cross_entropy = 1.068408
Epoch 256
Validation binary_cross_entropy = 1.134671
Epoch 257
Validation binary_cross_entropy = 1.128936
Epoch 258
Validation binary_cross_entropy = 1.092734
Epoch 259
Loss = 1.7808e-01, PNorm = 67.7298, GNorm = 1.2032, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 1.060179
Epoch 260
Validation binary_cross_entropy = 1.047304
Epoch 261
Validation binary_cross_entropy = 1.022848
Epoch 262
Validation binary_cross_entropy = 0.923475
Epoch 263
Validation binary_cross_entropy = 0.931697
Epoch 264
Loss = 2.0036e-01, PNorm = 67.7981, GNorm = 2.0497, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 0.831905
Epoch 265
Validation binary_cross_entropy = 0.845620
Epoch 266
Validation binary_cross_entropy = 0.843334
Epoch 267
Validation binary_cross_entropy = 0.744547
Epoch 268
Validation binary_cross_entropy = 0.699488
Epoch 269
Loss = 1.1784e-01, PNorm = 67.8713, GNorm = 2.3271, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 0.713096
Epoch 270
Validation binary_cross_entropy = 0.745419
Epoch 271
Validation binary_cross_entropy = 0.795106
Epoch 272
Validation binary_cross_entropy = 0.849127
Epoch 273
Validation binary_cross_entropy = 0.905021
Epoch 274
Loss = 5.1612e-02, PNorm = 67.9433, GNorm = 0.9625, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 0.913278
Epoch 275
Validation binary_cross_entropy = 0.957402
Epoch 276
Validation binary_cross_entropy = 0.996785
Epoch 277
Validation binary_cross_entropy = 1.122777
Epoch 278
Validation binary_cross_entropy = 1.158667
Epoch 279
Loss = 7.0621e-02, PNorm = 68.0026, GNorm = 3.4434, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 1.043021
Epoch 280
Validation binary_cross_entropy = 0.984485
Epoch 281
Validation binary_cross_entropy = 0.949563
Epoch 282
Validation binary_cross_entropy = 0.933258
Epoch 283
Validation binary_cross_entropy = 0.970764
Epoch 284
Loss = 1.0843e-01, PNorm = 68.0577, GNorm = 3.3861, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 0.949165
Epoch 285
Validation binary_cross_entropy = 0.904148
Epoch 286
Validation binary_cross_entropy = 0.869145
Epoch 287
Validation binary_cross_entropy = 0.875050
Epoch 288
Validation binary_cross_entropy = 0.931237
Epoch 289
Loss = 5.5501e-02, PNorm = 68.1115, GNorm = 1.0132, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 0.946712
Epoch 290
Validation binary_cross_entropy = 0.925204
Epoch 291
Validation binary_cross_entropy = 0.921000
Epoch 292
Validation binary_cross_entropy = 0.942452
Epoch 293
Validation binary_cross_entropy = 1.019716
Epoch 294
Loss = 8.9854e-02, PNorm = 68.1630, GNorm = 2.4913, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 1.182991
Epoch 295
Validation binary_cross_entropy = 0.987588
Epoch 296
Validation binary_cross_entropy = 0.935637
Epoch 297
Validation binary_cross_entropy = 0.931514
Epoch 298
Validation binary_cross_entropy = 0.905639
Epoch 299
Loss = 1.6295e-01, PNorm = 68.2377, GNorm = 1.3616, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 0.962049
Model 0 best validation binary_cross_entropy = 0.291358 on epoch 7
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.242493
Ensemble test binary_cross_entropy = 0.242493
Fold 1
Splitting data with seed 1
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.4, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=800, bias=False)
        (W_h): Linear(in_features=800, out_features=800, bias=False)
        (W_o): Linear(in_features=933, out_features=800, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.4, inplace=False)
    (1): Linear(in_features=3100, out_features=800, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.4, inplace=False)
    (4): Linear(in_features=800, out_features=800, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.4, inplace=False)
    (7): Linear(in_features=800, out_features=1, bias=True)
  )
)
Number of parameters = 4,627,201
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 0.289308
Epoch 1
Validation binary_cross_entropy = 0.308339
Epoch 2
Validation binary_cross_entropy = 0.745188
Epoch 3
Validation binary_cross_entropy = 0.310702
Epoch 4
Loss = 9.0577e-01, PNorm = 63.1501, GNorm = 10.4012, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 0.297426
Epoch 5
Validation binary_cross_entropy = 0.490699
Epoch 6
Validation binary_cross_entropy = 0.968219
Epoch 7
Validation binary_cross_entropy = 0.325563
Epoch 8
Validation binary_cross_entropy = 0.300375
Epoch 9
Loss = 7.2111e-01, PNorm = 63.2884, GNorm = 3.8343, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 0.761792
Epoch 10
Validation binary_cross_entropy = 0.812666
Epoch 11
Validation binary_cross_entropy = 0.304944
Epoch 12
Validation binary_cross_entropy = 0.370894
Epoch 13
Validation binary_cross_entropy = 0.966551
Epoch 14
Loss = 8.0977e-01, PNorm = 63.4289, GNorm = 5.2333, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 0.645028
Epoch 15
Validation binary_cross_entropy = 0.397379
Epoch 16
Validation binary_cross_entropy = 0.381264
Epoch 17
Validation binary_cross_entropy = 0.597862
Epoch 18
Validation binary_cross_entropy = 0.714151
Epoch 19
Loss = 6.9391e-01, PNorm = 63.5343, GNorm = 2.6725, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 0.397722
Epoch 20
Validation binary_cross_entropy = 0.372812
Epoch 21
Validation binary_cross_entropy = 0.542879
Epoch 22
Validation binary_cross_entropy = 0.724101
Epoch 23
Validation binary_cross_entropy = 0.487529
Epoch 24
Loss = 6.7108e-01, PNorm = 63.6206, GNorm = 3.4559, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 0.374869
Epoch 25
Validation binary_cross_entropy = 0.444442
Epoch 26
Validation binary_cross_entropy = 0.608225
Epoch 27
Validation binary_cross_entropy = 0.614403
Epoch 28
Validation binary_cross_entropy = 0.490906
Epoch 29
Loss = 6.3944e-01, PNorm = 63.6893, GNorm = 3.5720, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 0.468085
Epoch 30
Validation binary_cross_entropy = 0.588779
Epoch 31
Validation binary_cross_entropy = 0.543842
Epoch 32
Validation binary_cross_entropy = 0.345004
Epoch 33
Validation binary_cross_entropy = 0.393434
Epoch 34
Loss = 5.5787e-01, PNorm = 63.7513, GNorm = 2.3707, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 0.595307
Epoch 35
Validation binary_cross_entropy = 0.634573
Epoch 36
Validation binary_cross_entropy = 0.531896
Epoch 37
Validation binary_cross_entropy = 0.447408
Epoch 38
Validation binary_cross_entropy = 0.468821
Epoch 39
Loss = 6.9781e-01, PNorm = 63.8134, GNorm = 2.5434, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 0.568841
Epoch 40
Validation binary_cross_entropy = 0.593225
Epoch 41
Validation binary_cross_entropy = 0.540140
Epoch 42
Validation binary_cross_entropy = 0.507117
Epoch 43
Validation binary_cross_entropy = 0.587334
Epoch 44
Loss = 5.6295e-01, PNorm = 63.8690, GNorm = 3.0899, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 0.534266
Epoch 45
Validation binary_cross_entropy = 0.423658
Epoch 46
Validation binary_cross_entropy = 0.440844
Epoch 47
Validation binary_cross_entropy = 0.539909
Epoch 48
Validation binary_cross_entropy = 0.626085
Epoch 49
Loss = 6.2883e-01, PNorm = 63.9222, GNorm = 2.6025, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 0.597083
Epoch 50
Validation binary_cross_entropy = 0.487702
Epoch 51
Validation binary_cross_entropy = 0.489121
Epoch 52
Validation binary_cross_entropy = 0.583457
Epoch 53
Validation binary_cross_entropy = 0.522685
Epoch 54
Loss = 6.3797e-01, PNorm = 63.9804, GNorm = 2.1524, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 0.445645
Epoch 55
Validation binary_cross_entropy = 0.417933
Epoch 56
Validation binary_cross_entropy = 0.494369
Epoch 57
Validation binary_cross_entropy = 0.561727
Epoch 58
Validation binary_cross_entropy = 0.504752
Epoch 59
Loss = 5.6181e-01, PNorm = 64.0434, GNorm = 1.8386, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 0.511117
Epoch 60
Validation binary_cross_entropy = 0.512990
Epoch 61
Validation binary_cross_entropy = 0.410916
Epoch 62
Validation binary_cross_entropy = 0.425751
Epoch 63
Validation binary_cross_entropy = 0.399608
Epoch 64
Loss = 5.9481e-01, PNorm = 64.1133, GNorm = 1.7818, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 0.447627
Epoch 65
Validation binary_cross_entropy = 0.585543
Epoch 66
Validation binary_cross_entropy = 0.498073
Epoch 67
Validation binary_cross_entropy = 0.397328
Epoch 68
Validation binary_cross_entropy = 0.532170
Epoch 69
Loss = 5.3980e-01, PNorm = 64.1840, GNorm = 2.5068, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 0.617519
Epoch 70
Validation binary_cross_entropy = 0.357128
Epoch 71
Validation binary_cross_entropy = 0.362392
Epoch 72
Validation binary_cross_entropy = 0.513583
Epoch 73
Validation binary_cross_entropy = 0.645172
Epoch 74
Loss = 5.5613e-01, PNorm = 64.2727, GNorm = 1.7584, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 0.434225
Epoch 75
Validation binary_cross_entropy = 0.372060
Epoch 76
Validation binary_cross_entropy = 0.436520
Epoch 77
Validation binary_cross_entropy = 0.467176
Epoch 78
Validation binary_cross_entropy = 0.487819
Epoch 79
Loss = 4.6261e-01, PNorm = 64.3592, GNorm = 1.7598, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 0.466109
Epoch 80
Validation binary_cross_entropy = 0.632748
Epoch 81
Validation binary_cross_entropy = 0.433313
Epoch 82
Validation binary_cross_entropy = 0.361331
Epoch 83
Validation binary_cross_entropy = 0.537557
Epoch 84
Loss = 4.9723e-01, PNorm = 64.4520, GNorm = 2.1733, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 0.431467
Epoch 85
Validation binary_cross_entropy = 0.333543
Epoch 86
Validation binary_cross_entropy = 0.370174
Epoch 87
Validation binary_cross_entropy = 0.555276
Epoch 88
Validation binary_cross_entropy = 0.479054
Epoch 89
Loss = 4.5249e-01, PNorm = 64.5630, GNorm = 1.4847, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 0.329159
Epoch 90
Validation binary_cross_entropy = 0.380013
Epoch 91
Validation binary_cross_entropy = 0.409475
Epoch 92
Validation binary_cross_entropy = 0.364087
Epoch 93
Validation binary_cross_entropy = 0.476619
Epoch 94
Loss = 4.5156e-01, PNorm = 64.6781, GNorm = 1.3966, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 0.400531
Epoch 95
Validation binary_cross_entropy = 0.328011
Epoch 96
Validation binary_cross_entropy = 0.397281
Epoch 97
Validation binary_cross_entropy = 0.442510
Epoch 98
Validation binary_cross_entropy = 0.316867
Epoch 99
Loss = 3.9917e-01, PNorm = 64.7948, GNorm = 1.3384, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.381642
Epoch 100
Validation binary_cross_entropy = 0.340656
Epoch 101
Validation binary_cross_entropy = 0.343807
Epoch 102
Validation binary_cross_entropy = 0.409868
Epoch 103
Validation binary_cross_entropy = 0.331273
Epoch 104
Loss = 4.1234e-01, PNorm = 64.9178, GNorm = 2.1351, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.335633
Epoch 105
Validation binary_cross_entropy = 0.332484
Epoch 106
Validation binary_cross_entropy = 0.463478
Epoch 107
Validation binary_cross_entropy = 0.348700
Epoch 108
Validation binary_cross_entropy = 0.437580
Epoch 109
Loss = 3.7253e-01, PNorm = 65.0462, GNorm = 2.3925, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.392550
Epoch 110
Validation binary_cross_entropy = 0.364847
Epoch 111
Validation binary_cross_entropy = 0.437888
Epoch 112
Validation binary_cross_entropy = 0.386473
Epoch 113
Validation binary_cross_entropy = 0.712884
Epoch 114
Loss = 5.3870e-01, PNorm = 65.1742, GNorm = 4.7194, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.529560
Epoch 115
Validation binary_cross_entropy = 0.287942
Epoch 116
Validation binary_cross_entropy = 0.284855
Epoch 117
Validation binary_cross_entropy = 0.348772
Epoch 118
Validation binary_cross_entropy = 0.457036
Epoch 119
Loss = 4.3748e-01, PNorm = 65.3078, GNorm = 2.5577, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.349689
Epoch 120
Validation binary_cross_entropy = 0.314681
Epoch 121
Validation binary_cross_entropy = 0.337326
Epoch 122
Validation binary_cross_entropy = 0.412077
Epoch 123
Validation binary_cross_entropy = 0.388731
Epoch 124
Loss = 2.5967e-01, PNorm = 65.4271, GNorm = 1.8596, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.376822
Epoch 125
Validation binary_cross_entropy = 0.444676
Epoch 126
Validation binary_cross_entropy = 0.428776
Epoch 127
Validation binary_cross_entropy = 0.376223
Epoch 128
Validation binary_cross_entropy = 0.440184
Epoch 129
Loss = 3.0453e-01, PNorm = 65.5484, GNorm = 2.1337, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.404032
Epoch 130
Validation binary_cross_entropy = 0.382363
Epoch 131
Validation binary_cross_entropy = 0.387257
Epoch 132
Validation binary_cross_entropy = 0.453730
Epoch 133
Validation binary_cross_entropy = 0.388469
Epoch 134
Loss = 2.5706e-01, PNorm = 65.6563, GNorm = 3.7271, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.397460
Epoch 135
Validation binary_cross_entropy = 0.572592
Epoch 136
Validation binary_cross_entropy = 0.415203
Epoch 137
Validation binary_cross_entropy = 0.468749
Epoch 138
Validation binary_cross_entropy = 0.418509
Epoch 139
Loss = 2.2561e-01, PNorm = 65.7830, GNorm = 2.0765, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.517096
Epoch 140
Validation binary_cross_entropy = 0.434145
Epoch 141
Validation binary_cross_entropy = 0.436579
Epoch 142
Validation binary_cross_entropy = 0.438330
Epoch 143
Validation binary_cross_entropy = 0.451210
Epoch 144
Loss = 3.0741e-01, PNorm = 65.9027, GNorm = 1.8342, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.466161
Epoch 145
Validation binary_cross_entropy = 0.447815
Epoch 146
Validation binary_cross_entropy = 0.477796
Epoch 147
Validation binary_cross_entropy = 0.529831
Epoch 148
Validation binary_cross_entropy = 0.580780
Epoch 149
Loss = 2.0592e-01, PNorm = 66.0056, GNorm = 2.2377, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.587712
Epoch 150
Validation binary_cross_entropy = 0.592715
Epoch 151
Validation binary_cross_entropy = 0.583533
Epoch 152
Validation binary_cross_entropy = 0.536556
Epoch 153
Validation binary_cross_entropy = 0.515101
Epoch 154
Loss = 1.4351e-01, PNorm = 66.1022, GNorm = 1.7629, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.518773
Epoch 155
Validation binary_cross_entropy = 0.568467
Epoch 156
Validation binary_cross_entropy = 0.541344
Epoch 157
Validation binary_cross_entropy = 0.601563
Epoch 158
Validation binary_cross_entropy = 0.555374
Epoch 159
Loss = 2.1450e-01, PNorm = 66.2043, GNorm = 4.5361, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.572070
Epoch 160
Validation binary_cross_entropy = 0.543611
Epoch 161
Validation binary_cross_entropy = 0.550204
Epoch 162
Validation binary_cross_entropy = 0.446803
Epoch 163
Validation binary_cross_entropy = 0.501269
Epoch 164
Loss = 2.9160e-01, PNorm = 66.3242, GNorm = 4.4682, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.499625
Epoch 165
Validation binary_cross_entropy = 0.525995
Epoch 166
Validation binary_cross_entropy = 0.536545
Epoch 167
Validation binary_cross_entropy = 0.516530
Epoch 168
Validation binary_cross_entropy = 0.589170
Epoch 169
Loss = 2.2057e-01, PNorm = 66.4346, GNorm = 4.6080, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.590660
Epoch 170
Validation binary_cross_entropy = 0.562795
Epoch 171
Validation binary_cross_entropy = 0.546553
Epoch 172
Validation binary_cross_entropy = 0.573984
Epoch 173
Validation binary_cross_entropy = 0.605069
Epoch 174
Loss = 1.7019e-01, PNorm = 66.5300, GNorm = 1.2926, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 0.552804
Epoch 175
Validation binary_cross_entropy = 0.570557
Epoch 176
Validation binary_cross_entropy = 0.635902
Epoch 177
Validation binary_cross_entropy = 0.724224
Epoch 178
Validation binary_cross_entropy = 0.633745
Epoch 179
Loss = 2.8175e-01, PNorm = 66.6204, GNorm = 5.6135, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 0.553052
Epoch 180
Validation binary_cross_entropy = 0.524558
Epoch 181
Validation binary_cross_entropy = 0.545257
Epoch 182
Validation binary_cross_entropy = 0.465571
Epoch 183
Validation binary_cross_entropy = 0.468171
Epoch 184
Loss = 2.1440e-01, PNorm = 66.7146, GNorm = 1.2290, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 0.469993
Epoch 185
Validation binary_cross_entropy = 0.505197
Epoch 186
Validation binary_cross_entropy = 0.568072
Epoch 187
Validation binary_cross_entropy = 0.629627
Epoch 188
Validation binary_cross_entropy = 0.684553
Epoch 189
Loss = 2.8195e-01, PNorm = 66.8107, GNorm = 2.7529, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 0.707812
Epoch 190
Validation binary_cross_entropy = 0.688941
Epoch 191
Validation binary_cross_entropy = 0.684806
Epoch 192
Validation binary_cross_entropy = 0.663233
Epoch 193
Validation binary_cross_entropy = 0.645649
Epoch 194
Loss = 1.9146e-01, PNorm = 66.8932, GNorm = 3.0490, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 0.657359
Epoch 195
Validation binary_cross_entropy = 0.753763
Epoch 196
Validation binary_cross_entropy = 0.693986
Epoch 197
Validation binary_cross_entropy = 0.653661
Epoch 198
Validation binary_cross_entropy = 0.668118
Epoch 199
Loss = 1.1276e-01, PNorm = 66.9779, GNorm = 0.8511, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 0.694273
Epoch 200
Validation binary_cross_entropy = 0.756639
Epoch 201
Validation binary_cross_entropy = 0.799316
Epoch 202
Validation binary_cross_entropy = 0.719957
Epoch 203
Validation binary_cross_entropy = 0.793104
Epoch 204
Loss = 3.1371e-01, PNorm = 67.0476, GNorm = 3.4155, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 0.636993
Epoch 205
Validation binary_cross_entropy = 0.687180
Epoch 206
Validation binary_cross_entropy = 0.666155
Epoch 207
Validation binary_cross_entropy = 0.614224
Epoch 208
Validation binary_cross_entropy = 0.625573
Epoch 209
Loss = 1.3771e-01, PNorm = 67.1311, GNorm = 1.6311, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 0.673266
Epoch 210
Validation binary_cross_entropy = 0.760616
Epoch 211
Validation binary_cross_entropy = 0.844658
Epoch 212
Validation binary_cross_entropy = 0.758349
Epoch 213
Validation binary_cross_entropy = 0.674207
Epoch 214
Loss = 4.7765e-02, PNorm = 67.2067, GNorm = 0.3662, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 0.639954
Epoch 215
Validation binary_cross_entropy = 0.654130
Epoch 216
Validation binary_cross_entropy = 0.721411
Epoch 217
Validation binary_cross_entropy = 0.782650
Epoch 218
Validation binary_cross_entropy = 0.826679
Epoch 219
Loss = 1.8839e-01, PNorm = 67.2747, GNorm = 1.4270, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 0.826580
Epoch 220
Validation binary_cross_entropy = 0.857735
Epoch 221
Validation binary_cross_entropy = 0.909648
Epoch 222
Validation binary_cross_entropy = 0.921396
Epoch 223
Validation binary_cross_entropy = 0.859659
Epoch 224
Loss = 2.3139e-01, PNorm = 67.3357, GNorm = 3.3838, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 0.872573
Epoch 225
Validation binary_cross_entropy = 0.944735
Epoch 226
Validation binary_cross_entropy = 0.789036
Epoch 227
Validation binary_cross_entropy = 0.778644
Epoch 228
Validation binary_cross_entropy = 0.779214
Epoch 229
Loss = 6.5032e-02, PNorm = 67.3997, GNorm = 0.7788, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 0.818322
Epoch 230
Validation binary_cross_entropy = 0.835143
Epoch 231
Validation binary_cross_entropy = 0.828347
Epoch 232
Validation binary_cross_entropy = 0.844112
Epoch 233
Validation binary_cross_entropy = 0.858756
Epoch 234
Loss = 6.4748e-02, PNorm = 67.4588, GNorm = 1.2652, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 0.882232
Epoch 235
Validation binary_cross_entropy = 0.945686
Epoch 236
Validation binary_cross_entropy = 1.016312
Epoch 237
Validation binary_cross_entropy = 1.039401
Epoch 238
Validation binary_cross_entropy = 1.013759
Epoch 239
Loss = 7.7572e-02, PNorm = 67.5192, GNorm = 2.9065, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 1.006788
Epoch 240
Validation binary_cross_entropy = 1.060846
Epoch 241
Validation binary_cross_entropy = 1.133138
Epoch 242
Validation binary_cross_entropy = 0.855320
Epoch 243
Validation binary_cross_entropy = 0.743544
Epoch 244
Loss = 2.1591e-01, PNorm = 67.5761, GNorm = 2.7385, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 0.687889
Epoch 245
Validation binary_cross_entropy = 0.700435
Epoch 246
Validation binary_cross_entropy = 0.726985
Epoch 247
Validation binary_cross_entropy = 0.696845
Epoch 248
Validation binary_cross_entropy = 0.688697
Epoch 249
Loss = 1.3927e-01, PNorm = 67.6470, GNorm = 1.8450, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 0.716639
Epoch 250
Validation binary_cross_entropy = 0.759529
Epoch 251
Validation binary_cross_entropy = 0.839970
Epoch 252
Validation binary_cross_entropy = 0.912774
Epoch 253
Validation binary_cross_entropy = 0.968920
Epoch 254
Loss = 5.6834e-02, PNorm = 67.7127, GNorm = 1.4193, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 1.024158
Epoch 255
Validation binary_cross_entropy = 1.100507
Epoch 256
Validation binary_cross_entropy = 1.124451
Epoch 257
Validation binary_cross_entropy = 1.151587
Epoch 258
Validation binary_cross_entropy = 1.168363
Epoch 259
Loss = 2.1847e-02, PNorm = 67.7691, GNorm = 0.3959, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 1.170577
Epoch 260
Validation binary_cross_entropy = 1.150499
Epoch 261
Validation binary_cross_entropy = 1.137046
Epoch 262
Validation binary_cross_entropy = 1.142108
Epoch 263
Validation binary_cross_entropy = 1.081942
Epoch 264
Loss = 3.4890e-02, PNorm = 67.8215, GNorm = 1.0332, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 1.185257
Epoch 265
Validation binary_cross_entropy = 1.208048
Epoch 266
Validation binary_cross_entropy = 1.112380
Epoch 267
Validation binary_cross_entropy = 1.048141
Epoch 268
Validation binary_cross_entropy = 0.969296
Epoch 269
Loss = 4.3590e-02, PNorm = 67.8811, GNorm = 2.0567, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 0.966447
Epoch 270
Validation binary_cross_entropy = 0.920465
Epoch 271
Validation binary_cross_entropy = 0.913264
Epoch 272
Validation binary_cross_entropy = 1.067521
Epoch 273
Validation binary_cross_entropy = 1.069868
Epoch 274
Loss = 2.7767e-02, PNorm = 67.9443, GNorm = 0.3622, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 0.933185
Epoch 275
Validation binary_cross_entropy = 0.899960
Epoch 276
Validation binary_cross_entropy = 0.886046
Epoch 277
Validation binary_cross_entropy = 0.918143
Epoch 278
Validation binary_cross_entropy = 0.954700
Epoch 279
Loss = 8.0904e-02, PNorm = 68.0034, GNorm = 1.4508, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 0.899228
Epoch 280
Validation binary_cross_entropy = 0.874867
Epoch 281
Validation binary_cross_entropy = 0.897311
Epoch 282
Validation binary_cross_entropy = 0.940683
Epoch 283
Validation binary_cross_entropy = 1.050726
Epoch 284
Loss = 5.3572e-02, PNorm = 68.0619, GNorm = 1.6224, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 1.137102
Epoch 285
Validation binary_cross_entropy = 1.132738
Epoch 286
Validation binary_cross_entropy = 1.153462
Epoch 287
Validation binary_cross_entropy = 1.176990
Epoch 288
Validation binary_cross_entropy = 1.233647
Epoch 289
Loss = 2.9371e-02, PNorm = 68.1240, GNorm = 0.7905, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 1.296764
Epoch 290
Validation binary_cross_entropy = 1.281099
Epoch 291
Validation binary_cross_entropy = 1.198955
Epoch 292
Validation binary_cross_entropy = 1.186077
Epoch 293
Validation binary_cross_entropy = 1.252288
Epoch 294
Loss = 3.9203e-02, PNorm = 68.1847, GNorm = 1.4202, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 1.259666
Epoch 295
Validation binary_cross_entropy = 1.139686
Epoch 296
Validation binary_cross_entropy = 1.104261
Epoch 297
Validation binary_cross_entropy = 1.093130
Epoch 298
Validation binary_cross_entropy = 1.075757
Epoch 299
Loss = 4.9867e-02, PNorm = 68.2485, GNorm = 0.8541, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 1.134324
Model 0 best validation binary_cross_entropy = 0.284855 on epoch 116
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.198337
Ensemble test binary_cross_entropy = 0.198337
Fold 2
Splitting data with seed 2
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.4, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=800, bias=False)
        (W_h): Linear(in_features=800, out_features=800, bias=False)
        (W_o): Linear(in_features=933, out_features=800, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.4, inplace=False)
    (1): Linear(in_features=3100, out_features=800, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.4, inplace=False)
    (4): Linear(in_features=800, out_features=800, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.4, inplace=False)
    (7): Linear(in_features=800, out_features=1, bias=True)
  )
)
Number of parameters = 4,627,201
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 0.289617
Epoch 1
Validation binary_cross_entropy = 0.304917
Epoch 2
Validation binary_cross_entropy = 0.601288
Epoch 3
Validation binary_cross_entropy = 0.420122
Epoch 4
Loss = 7.6577e-01, PNorm = 63.1502, GNorm = 7.0863, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 0.297528
Epoch 5
Validation binary_cross_entropy = 0.422881
Epoch 6
Validation binary_cross_entropy = 0.756407
Epoch 7
Validation binary_cross_entropy = 0.303197
Epoch 8
Validation binary_cross_entropy = 0.332482
Epoch 9
Loss = 8.5060e-01, PNorm = 63.2876, GNorm = 5.5403, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 0.913284
Epoch 10
Validation binary_cross_entropy = 0.310293
Epoch 11
Validation binary_cross_entropy = 0.325268
Epoch 12
Validation binary_cross_entropy = 0.718814
Epoch 13
Validation binary_cross_entropy = 0.341210
Epoch 14
Loss = 8.1144e-01, PNorm = 63.4388, GNorm = 5.7765, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 0.346163
Epoch 15
Validation binary_cross_entropy = 0.693508
Epoch 16
Validation binary_cross_entropy = 0.361234
Epoch 17
Validation binary_cross_entropy = 0.303745
Epoch 18
Validation binary_cross_entropy = 0.468845
Epoch 19
Loss = 6.2844e-01, PNorm = 63.5597, GNorm = 5.5798, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 0.659127
Epoch 20
Validation binary_cross_entropy = 0.324386
Epoch 21
Validation binary_cross_entropy = 0.304827
Epoch 22
Validation binary_cross_entropy = 0.408427
Epoch 23
Validation binary_cross_entropy = 0.990125
Epoch 24
Loss = 7.3082e-01, PNorm = 63.6433, GNorm = 5.6986, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 0.674218
Epoch 25
Validation binary_cross_entropy = 0.367297
Epoch 26
Validation binary_cross_entropy = 0.418726
Epoch 27
Validation binary_cross_entropy = 0.720029
Epoch 28
Validation binary_cross_entropy = 0.713908
Epoch 29
Loss = 5.8398e-01, PNorm = 63.7123, GNorm = 2.9650, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 0.530297
Epoch 30
Validation binary_cross_entropy = 0.520336
Epoch 31
Validation binary_cross_entropy = 0.568638
Epoch 32
Validation binary_cross_entropy = 0.549944
Epoch 33
Validation binary_cross_entropy = 0.533363
Epoch 34
Loss = 6.9430e-01, PNorm = 63.7777, GNorm = 2.6115, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 0.541354
Epoch 35
Validation binary_cross_entropy = 0.560324
Epoch 36
Validation binary_cross_entropy = 0.444775
Epoch 37
Validation binary_cross_entropy = 0.415333
Epoch 38
Validation binary_cross_entropy = 0.591142
Epoch 39
Loss = 6.3802e-01, PNorm = 63.8303, GNorm = 3.3378, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 0.675578
Epoch 40
Validation binary_cross_entropy = 0.511834
Epoch 41
Validation binary_cross_entropy = 0.443027
Epoch 42
Validation binary_cross_entropy = 0.516000
Epoch 43
Validation binary_cross_entropy = 0.583309
Epoch 44
Loss = 6.6222e-01, PNorm = 63.8797, GNorm = 3.1191, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 0.601022
Epoch 45
Validation binary_cross_entropy = 0.431168
Epoch 46
Validation binary_cross_entropy = 0.402526
Epoch 47
Validation binary_cross_entropy = 0.422764
Epoch 48
Validation binary_cross_entropy = 0.484612
Epoch 49
Loss = 6.1539e-01, PNorm = 63.9383, GNorm = 2.3557, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 0.517515
Epoch 50
Validation binary_cross_entropy = 0.467650
Epoch 51
Validation binary_cross_entropy = 0.433795
Epoch 52
Validation binary_cross_entropy = 0.488805
Epoch 53
Validation binary_cross_entropy = 0.514093
Epoch 54
Loss = 5.8996e-01, PNorm = 63.9949, GNorm = 2.0851, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 0.531979
Epoch 55
Validation binary_cross_entropy = 0.505982
Epoch 56
Validation binary_cross_entropy = 0.417945
Epoch 57
Validation binary_cross_entropy = 0.415940
Epoch 58
Validation binary_cross_entropy = 0.375466
Epoch 59
Loss = 6.2153e-01, PNorm = 64.0547, GNorm = 1.6428, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 0.461701
Epoch 60
Validation binary_cross_entropy = 0.625850
Epoch 61
Validation binary_cross_entropy = 0.607951
Epoch 62
Validation binary_cross_entropy = 0.487565
Epoch 63
Validation binary_cross_entropy = 0.463475
Epoch 64
Loss = 5.3557e-01, PNorm = 64.1314, GNorm = 2.1343, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 0.475087
Epoch 65
Validation binary_cross_entropy = 0.396542
Epoch 66
Validation binary_cross_entropy = 0.415175
Epoch 67
Validation binary_cross_entropy = 0.509714
Epoch 68
Validation binary_cross_entropy = 0.438522
Epoch 69
Loss = 5.2263e-01, PNorm = 64.2227, GNorm = 1.5761, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 0.438391
Epoch 70
Validation binary_cross_entropy = 0.499057
Epoch 71
Validation binary_cross_entropy = 0.364971
Epoch 72
Validation binary_cross_entropy = 0.558006
Epoch 73
Validation binary_cross_entropy = 0.470181
Epoch 74
Loss = 5.0133e-01, PNorm = 64.3167, GNorm = 1.3991, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 0.387823
Epoch 75
Validation binary_cross_entropy = 0.368294
Epoch 76
Validation binary_cross_entropy = 0.414867
Epoch 77
Validation binary_cross_entropy = 0.493050
Epoch 78
Validation binary_cross_entropy = 0.328329
Epoch 79
Loss = 5.2386e-01, PNorm = 64.4258, GNorm = 1.7192, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 0.390897
Epoch 80
Validation binary_cross_entropy = 0.488785
Epoch 81
Validation binary_cross_entropy = 0.270032
Epoch 82
Validation binary_cross_entropy = 0.436148
Epoch 83
Validation binary_cross_entropy = 0.542536
Epoch 84
Loss = 5.1344e-01, PNorm = 64.5311, GNorm = 1.3049, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 0.401678
Epoch 85
Validation binary_cross_entropy = 0.380955
Epoch 86
Validation binary_cross_entropy = 0.418707
Epoch 87
Validation binary_cross_entropy = 0.469891
Epoch 88
Validation binary_cross_entropy = 0.428597
Epoch 89
Loss = 5.1390e-01, PNorm = 64.6448, GNorm = 1.6964, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 0.346294
Epoch 90
Validation binary_cross_entropy = 0.355845
Epoch 91
Validation binary_cross_entropy = 0.488675
Epoch 92
Validation binary_cross_entropy = 0.337905
Epoch 93
Validation binary_cross_entropy = 0.352543
Epoch 94
Loss = 3.7831e-01, PNorm = 64.7592, GNorm = 1.8561, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 0.339665
Epoch 95
Validation binary_cross_entropy = 0.359025
Epoch 96
Validation binary_cross_entropy = 0.303552
Epoch 97
Validation binary_cross_entropy = 0.354828
Epoch 98
Validation binary_cross_entropy = 0.390987
Epoch 99
Loss = 3.7516e-01, PNorm = 64.8644, GNorm = 2.0720, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.314549
Epoch 100
Validation binary_cross_entropy = 0.422985
Epoch 101
Validation binary_cross_entropy = 0.389920
Epoch 102
Validation binary_cross_entropy = 0.365315
Epoch 103
Validation binary_cross_entropy = 0.429874
Epoch 104
Loss = 4.3461e-01, PNorm = 64.9902, GNorm = 1.4622, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.309185
Epoch 105
Validation binary_cross_entropy = 0.398383
Epoch 106
Validation binary_cross_entropy = 0.423090
Epoch 107
Validation binary_cross_entropy = 0.304349
Epoch 108
Validation binary_cross_entropy = 0.327756
Epoch 109
Loss = 3.9548e-01, PNorm = 65.1263, GNorm = 2.6429, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.373369
Epoch 110
Validation binary_cross_entropy = 0.360013
Epoch 111
Validation binary_cross_entropy = 0.585442
Epoch 112
Validation binary_cross_entropy = 0.465499
Epoch 113
Validation binary_cross_entropy = 0.406105
Epoch 114
Loss = 2.9206e-01, PNorm = 65.2615, GNorm = 1.6956, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.341595
Epoch 115
Validation binary_cross_entropy = 0.342887
Epoch 116
Validation binary_cross_entropy = 0.409773
Epoch 117
Validation binary_cross_entropy = 0.348576
Epoch 118
Validation binary_cross_entropy = 0.447500
Epoch 119
Loss = 3.0793e-01, PNorm = 65.4015, GNorm = 2.2321, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.439217
Epoch 120
Validation binary_cross_entropy = 0.368593
Epoch 121
Validation binary_cross_entropy = 0.426201
Epoch 122
Validation binary_cross_entropy = 0.394298
Epoch 123
Validation binary_cross_entropy = 0.365323
Epoch 124
Loss = 1.8249e-01, PNorm = 65.5519, GNorm = 1.0571, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.482644
Epoch 125
Validation binary_cross_entropy = 0.548106
Epoch 126
Validation binary_cross_entropy = 0.406232
Epoch 127
Validation binary_cross_entropy = 0.504324
Epoch 128
Validation binary_cross_entropy = 0.385373
Epoch 129
Loss = 3.2431e-01, PNorm = 65.6953, GNorm = 4.5602, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.474217
Epoch 130
Validation binary_cross_entropy = 0.518473
Epoch 131
Validation binary_cross_entropy = 0.352604
Epoch 132
Validation binary_cross_entropy = 0.368539
Epoch 133
Validation binary_cross_entropy = 0.583950
Epoch 134
Loss = 2.4005e-01, PNorm = 65.8387, GNorm = 1.7376, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.476216
Epoch 135
Validation binary_cross_entropy = 0.413791
Epoch 136
Validation binary_cross_entropy = 0.412420
Epoch 137
Validation binary_cross_entropy = 0.433675
Epoch 138
Validation binary_cross_entropy = 0.467763
Epoch 139
Loss = 3.2667e-01, PNorm = 65.9711, GNorm = 1.2877, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.376226
Epoch 140
Validation binary_cross_entropy = 0.396020
Epoch 141
Validation binary_cross_entropy = 0.548683
Epoch 142
Validation binary_cross_entropy = 0.477305
Epoch 143
Validation binary_cross_entropy = 0.432081
Epoch 144
Loss = 1.8912e-01, PNorm = 66.1027, GNorm = 2.6414, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.447471
Epoch 145
Validation binary_cross_entropy = 0.506178
Epoch 146
Validation binary_cross_entropy = 0.505655
Epoch 147
Validation binary_cross_entropy = 0.499793
Epoch 148
Validation binary_cross_entropy = 0.501598
Epoch 149
Loss = 1.8092e-01, PNorm = 66.2169, GNorm = 1.1772, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.529852
Epoch 150
Validation binary_cross_entropy = 0.546075
Epoch 151
Validation binary_cross_entropy = 0.540012
Epoch 152
Validation binary_cross_entropy = 0.537561
Epoch 153
Validation binary_cross_entropy = 0.575755
Epoch 154
Loss = 2.2818e-01, PNorm = 66.3099, GNorm = 1.2615, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.528691
Epoch 155
Validation binary_cross_entropy = 0.535657
Epoch 156
Validation binary_cross_entropy = 0.569985
Epoch 157
Validation binary_cross_entropy = 0.612173
Epoch 158
Validation binary_cross_entropy = 0.560559
Epoch 159
Loss = 2.5113e-01, PNorm = 66.4037, GNorm = 5.9152, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.550427
Epoch 160
Validation binary_cross_entropy = 0.531274
Epoch 161
Validation binary_cross_entropy = 0.502541
Epoch 162
Validation binary_cross_entropy = 0.477752
Epoch 163
Validation binary_cross_entropy = 0.479849
Epoch 164
Loss = 1.7858e-01, PNorm = 66.5064, GNorm = 2.3603, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.476712
Epoch 165
Validation binary_cross_entropy = 0.492389
Epoch 166
Validation binary_cross_entropy = 0.511867
Epoch 167
Validation binary_cross_entropy = 0.559460
Epoch 168
Validation binary_cross_entropy = 0.568172
Epoch 169
Loss = 2.3865e-01, PNorm = 66.6028, GNorm = 2.5571, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.573370
Epoch 170
Validation binary_cross_entropy = 0.582988
Epoch 171
Validation binary_cross_entropy = 0.611908
Epoch 172
Validation binary_cross_entropy = 0.662162
Epoch 173
Validation binary_cross_entropy = 0.648682
Epoch 174
Loss = 1.1630e-01, PNorm = 66.6882, GNorm = 2.2408, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 0.660404
Epoch 175
Validation binary_cross_entropy = 0.663350
Epoch 176
Validation binary_cross_entropy = 0.688907
Epoch 177
Validation binary_cross_entropy = 0.699619
Epoch 178
Validation binary_cross_entropy = 0.737166
Epoch 179
Loss = 1.7644e-01, PNorm = 66.7722, GNorm = 1.9250, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 0.753317
Epoch 180
Validation binary_cross_entropy = 0.752408
Epoch 181
Validation binary_cross_entropy = 0.727404
Epoch 182
Validation binary_cross_entropy = 0.722525
Epoch 183
Validation binary_cross_entropy = 0.697629
Epoch 184
Loss = 1.6365e-01, PNorm = 66.8553, GNorm = 0.9590, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 0.704325
Epoch 185
Validation binary_cross_entropy = 0.684196
Epoch 186
Validation binary_cross_entropy = 0.661657
Epoch 187
Validation binary_cross_entropy = 0.680440
Epoch 188
Validation binary_cross_entropy = 0.660377
Epoch 189
Loss = 1.4425e-01, PNorm = 66.9331, GNorm = 2.3267, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 0.660017
Epoch 190
Validation binary_cross_entropy = 0.669937
Epoch 191
Validation binary_cross_entropy = 0.702215
Epoch 192
Validation binary_cross_entropy = 0.741260
Epoch 193
Validation binary_cross_entropy = 0.767917
Epoch 194
Loss = 9.1339e-02, PNorm = 67.0233, GNorm = 2.3180, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 0.801477
Epoch 195
Validation binary_cross_entropy = 0.857411
Epoch 196
Validation binary_cross_entropy = 0.707694
Epoch 197
Validation binary_cross_entropy = 0.659324
Epoch 198
Validation binary_cross_entropy = 0.619669
Epoch 199
Loss = 1.4107e-01, PNorm = 67.1039, GNorm = 1.0598, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 0.607717
Epoch 200
Validation binary_cross_entropy = 0.597727
Epoch 201
Validation binary_cross_entropy = 0.631146
Epoch 202
Validation binary_cross_entropy = 0.660322
Epoch 203
Validation binary_cross_entropy = 0.783280
Epoch 204
Loss = 1.8524e-01, PNorm = 67.1870, GNorm = 4.2221, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 0.797089
Epoch 205
Validation binary_cross_entropy = 0.795312
Epoch 206
Validation binary_cross_entropy = 0.851332
Epoch 207
Validation binary_cross_entropy = 0.918376
Epoch 208
Validation binary_cross_entropy = 0.792847
Epoch 209
Loss = 4.6530e-02, PNorm = 67.2745, GNorm = 0.6507, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 0.751124
Epoch 210
Validation binary_cross_entropy = 0.682671
Epoch 211
Validation binary_cross_entropy = 0.642507
Epoch 212
Validation binary_cross_entropy = 0.636322
Epoch 213
Validation binary_cross_entropy = 0.634993
Epoch 214
Loss = 1.8258e-01, PNorm = 67.3645, GNorm = 1.3987, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 0.644929
Epoch 215
Validation binary_cross_entropy = 0.612446
Epoch 216
Validation binary_cross_entropy = 0.651779
Epoch 217
Validation binary_cross_entropy = 0.687839
Epoch 218
Validation binary_cross_entropy = 0.687274
Epoch 219
Loss = 1.3855e-01, PNorm = 67.4513, GNorm = 1.8090, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 0.788704
Epoch 220
Validation binary_cross_entropy = 0.675384
Epoch 221
Validation binary_cross_entropy = 0.712676
Epoch 222
Validation binary_cross_entropy = 0.668932
Epoch 223
Validation binary_cross_entropy = 0.587459
Epoch 224
Loss = 9.2291e-02, PNorm = 67.5186, GNorm = 0.8868, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 0.575271
Epoch 225
Validation binary_cross_entropy = 0.580232
Epoch 226
Validation binary_cross_entropy = 0.606595
Epoch 227
Validation binary_cross_entropy = 0.668242
Epoch 228
Validation binary_cross_entropy = 0.694999
Epoch 229
Loss = 8.1139e-02, PNorm = 67.5856, GNorm = 1.0513, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 0.729251
Epoch 230
Validation binary_cross_entropy = 0.772452
Epoch 231
Validation binary_cross_entropy = 0.809136
Epoch 232
Validation binary_cross_entropy = 0.830582
Epoch 233
Validation binary_cross_entropy = 0.828586
Epoch 234
Loss = 5.7201e-02, PNorm = 67.6479, GNorm = 2.1207, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 0.850209
Epoch 235
Validation binary_cross_entropy = 0.885590
Epoch 236
Validation binary_cross_entropy = 0.929206
Epoch 237
Validation binary_cross_entropy = 0.936283
Epoch 238
Validation binary_cross_entropy = 0.953206
Epoch 239
Loss = 1.3627e-01, PNorm = 67.7015, GNorm = 4.9084, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 0.946843
Epoch 240
Validation binary_cross_entropy = 0.904491
Epoch 241
Validation binary_cross_entropy = 0.864527
Epoch 242
Validation binary_cross_entropy = 0.836601
Epoch 243
Validation binary_cross_entropy = 0.824363
Epoch 244
Loss = 1.0365e-01, PNorm = 67.7502, GNorm = 1.5948, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 0.816309
Epoch 245
Validation binary_cross_entropy = 0.805920
Epoch 246
Validation binary_cross_entropy = 0.816564
Epoch 247
Validation binary_cross_entropy = 0.832855
Epoch 248
Validation binary_cross_entropy = 0.839924
Epoch 249
Loss = 9.9414e-02, PNorm = 67.8105, GNorm = 0.8862, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 0.858812
Epoch 250
Validation binary_cross_entropy = 0.884752
Epoch 251
Validation binary_cross_entropy = 0.943205
Epoch 252
Validation binary_cross_entropy = 0.871087
Epoch 253
Validation binary_cross_entropy = 0.843792
Epoch 254
Loss = 5.8441e-02, PNorm = 67.8776, GNorm = 1.2369, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 0.834478
Epoch 255
Validation binary_cross_entropy = 0.823732
Epoch 256
Validation binary_cross_entropy = 0.827302
Epoch 257
Validation binary_cross_entropy = 0.841992
Epoch 258
Validation binary_cross_entropy = 0.844975
Epoch 259
Loss = 3.9627e-02, PNorm = 67.9412, GNorm = 1.5323, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 0.861016
Epoch 260
Validation binary_cross_entropy = 0.891484
Epoch 261
Validation binary_cross_entropy = 0.930675
Epoch 262
Validation binary_cross_entropy = 0.976460
Epoch 263
Validation binary_cross_entropy = 1.013079
Epoch 264
Loss = 2.5555e-02, PNorm = 67.9924, GNorm = 0.7473, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 1.048298
Epoch 265
Validation binary_cross_entropy = 1.085888
Epoch 266
Validation binary_cross_entropy = 1.136242
Epoch 267
Validation binary_cross_entropy = 1.138663
Epoch 268
Validation binary_cross_entropy = 1.130452
Epoch 269
Loss = 9.2432e-02, PNorm = 68.0279, GNorm = 0.8972, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 1.149886
Epoch 270
Validation binary_cross_entropy = 1.094458
Epoch 271
Validation binary_cross_entropy = 1.043505
Epoch 272
Validation binary_cross_entropy = 1.029505
Epoch 273
Validation binary_cross_entropy = 1.020263
Epoch 274
Loss = 5.9578e-02, PNorm = 68.0705, GNorm = 2.8084, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 1.024821
Epoch 275
Validation binary_cross_entropy = 1.015292
Epoch 276
Validation binary_cross_entropy = 1.076390
Epoch 277
Validation binary_cross_entropy = 1.029383
Epoch 278
Validation binary_cross_entropy = 0.931234
Epoch 279
Loss = 7.1647e-02, PNorm = 68.1198, GNorm = 1.1033, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 0.881514
Epoch 280
Validation binary_cross_entropy = 0.879637
Epoch 281
Validation binary_cross_entropy = 0.899193
Epoch 282
Validation binary_cross_entropy = 0.932059
Epoch 283
Validation binary_cross_entropy = 1.008843
Epoch 284
Loss = 8.2893e-02, PNorm = 68.1745, GNorm = 1.4582, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 1.105471
Epoch 285
Validation binary_cross_entropy = 1.096739
Epoch 286
Validation binary_cross_entropy = 1.113582
Epoch 287
Validation binary_cross_entropy = 1.121947
Epoch 288
Validation binary_cross_entropy = 1.177566
Epoch 289
Loss = 2.7953e-02, PNorm = 68.2217, GNorm = 2.0306, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 1.314124
Epoch 290
Validation binary_cross_entropy = 1.178311
Epoch 291
Validation binary_cross_entropy = 1.073959
Epoch 292
Validation binary_cross_entropy = 1.002623
Epoch 293
Validation binary_cross_entropy = 0.782546
Epoch 294
Loss = 2.4711e-01, PNorm = 68.2811, GNorm = 1.2912, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 0.702418
Epoch 295
Validation binary_cross_entropy = 0.641909
Epoch 296
Validation binary_cross_entropy = 0.570542
Epoch 297
Validation binary_cross_entropy = 0.546948
Epoch 298
Validation binary_cross_entropy = 0.553379
Epoch 299
Loss = 1.2045e-01, PNorm = 68.3430, GNorm = 0.9221, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 0.574303
Model 0 best validation binary_cross_entropy = 0.270032 on epoch 81
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.205699
Ensemble test binary_cross_entropy = 0.205699
Fold 3
Splitting data with seed 3
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.4, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=800, bias=False)
        (W_h): Linear(in_features=800, out_features=800, bias=False)
        (W_o): Linear(in_features=933, out_features=800, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.4, inplace=False)
    (1): Linear(in_features=3100, out_features=800, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.4, inplace=False)
    (4): Linear(in_features=800, out_features=800, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.4, inplace=False)
    (7): Linear(in_features=800, out_features=1, bias=True)
  )
)
Number of parameters = 4,627,201
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 0.292552
Epoch 1
Validation binary_cross_entropy = 0.314568
Epoch 2
Validation binary_cross_entropy = 0.721326
Epoch 3
Validation binary_cross_entropy = 0.305099
Epoch 4
Loss = 8.1420e-01, PNorm = 63.1529, GNorm = 6.9891, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 0.290179
Epoch 5
Validation binary_cross_entropy = 0.581570
Epoch 6
Validation binary_cross_entropy = 0.592783
Epoch 7
Validation binary_cross_entropy = 0.288288
Epoch 8
Validation binary_cross_entropy = 0.323829
Epoch 9
Loss = 6.8788e-01, PNorm = 63.2945, GNorm = 4.8888, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 0.737028
Epoch 10
Validation binary_cross_entropy = 0.332306
Epoch 11
Validation binary_cross_entropy = 0.324212
Epoch 12
Validation binary_cross_entropy = 0.496599
Epoch 13
Validation binary_cross_entropy = 0.373763
Epoch 14
Loss = 7.4570e-01, PNorm = 63.4460, GNorm = 4.5970, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 0.318944
Epoch 15
Validation binary_cross_entropy = 0.560535
Epoch 16
Validation binary_cross_entropy = 0.523742
Epoch 17
Validation binary_cross_entropy = 0.319813
Epoch 18
Validation binary_cross_entropy = 0.371191
Epoch 19
Loss = 7.1945e-01, PNorm = 63.5611, GNorm = 3.8456, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 0.643111
Epoch 20
Validation binary_cross_entropy = 0.409281
Epoch 21
Validation binary_cross_entropy = 0.307546
Epoch 22
Validation binary_cross_entropy = 0.365077
Epoch 23
Validation binary_cross_entropy = 0.621935
Epoch 24
Loss = 6.8321e-01, PNorm = 63.6446, GNorm = 4.3268, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 0.424910
Epoch 25
Validation binary_cross_entropy = 0.333438
Epoch 26
Validation binary_cross_entropy = 0.416331
Epoch 27
Validation binary_cross_entropy = 0.688589
Epoch 28
Validation binary_cross_entropy = 0.508631
Epoch 29
Loss = 5.8375e-01, PNorm = 63.7217, GNorm = 2.2870, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 0.449368
Epoch 30
Validation binary_cross_entropy = 0.498492
Epoch 31
Validation binary_cross_entropy = 0.588410
Epoch 32
Validation binary_cross_entropy = 0.406770
Epoch 33
Validation binary_cross_entropy = 0.467535
Epoch 34
Loss = 6.7340e-01, PNorm = 63.7850, GNorm = 2.9855, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 0.831251
Epoch 35
Validation binary_cross_entropy = 0.867948
Epoch 36
Validation binary_cross_entropy = 0.526878
Epoch 37
Validation binary_cross_entropy = 0.382611
Epoch 38
Validation binary_cross_entropy = 0.437268
Epoch 39
Loss = 6.4439e-01, PNorm = 63.8663, GNorm = 2.9600, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 0.646669
Epoch 40
Validation binary_cross_entropy = 0.688491
Epoch 41
Validation binary_cross_entropy = 0.555366
Epoch 42
Validation binary_cross_entropy = 0.426560
Epoch 43
Validation binary_cross_entropy = 0.391114
Epoch 44
Loss = 6.4086e-01, PNorm = 63.9220, GNorm = 2.3617, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 0.460353
Epoch 45
Validation binary_cross_entropy = 0.567913
Epoch 46
Validation binary_cross_entropy = 0.612058
Epoch 47
Validation binary_cross_entropy = 0.529652
Epoch 48
Validation binary_cross_entropy = 0.482848
Epoch 49
Loss = 5.5242e-01, PNorm = 63.9805, GNorm = 1.8685, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 0.466197
Epoch 50
Validation binary_cross_entropy = 0.488856
Epoch 51
Validation binary_cross_entropy = 0.517385
Epoch 52
Validation binary_cross_entropy = 0.512792
Epoch 53
Validation binary_cross_entropy = 0.483654
Epoch 54
Loss = 5.3956e-01, PNorm = 64.0228, GNorm = 1.8113, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 0.495857
Epoch 55
Validation binary_cross_entropy = 0.578116
Epoch 56
Validation binary_cross_entropy = 0.611870
Epoch 57
Validation binary_cross_entropy = 0.549744
Epoch 58
Validation binary_cross_entropy = 0.485417
Epoch 59
Loss = 6.3925e-01, PNorm = 64.0839, GNorm = 1.9920, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 0.446963
Epoch 60
Validation binary_cross_entropy = 0.538598
Epoch 61
Validation binary_cross_entropy = 0.583346
Epoch 62
Validation binary_cross_entropy = 0.561089
Epoch 63
Validation binary_cross_entropy = 0.496523
Epoch 64
Loss = 5.6670e-01, PNorm = 64.1440, GNorm = 1.5366, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 0.472133
Epoch 65
Validation binary_cross_entropy = 0.485509
Epoch 66
Validation binary_cross_entropy = 0.524734
Epoch 67
Validation binary_cross_entropy = 0.580696
Epoch 68
Validation binary_cross_entropy = 0.499647
Epoch 69
Loss = 5.4936e-01, PNorm = 64.2028, GNorm = 1.5289, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 0.460163
Epoch 70
Validation binary_cross_entropy = 0.507534
Epoch 71
Validation binary_cross_entropy = 0.564477
Epoch 72
Validation binary_cross_entropy = 0.439626
Epoch 73
Validation binary_cross_entropy = 0.401142
Epoch 74
Loss = 5.3582e-01, PNorm = 64.2652, GNorm = 1.6479, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 0.489256
Epoch 75
Validation binary_cross_entropy = 0.612832
Epoch 76
Validation binary_cross_entropy = 0.634315
Epoch 77
Validation binary_cross_entropy = 0.465726
Epoch 78
Validation binary_cross_entropy = 0.378660
Epoch 79
Loss = 5.7250e-01, PNorm = 64.3428, GNorm = 2.6554, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 0.432455
Epoch 80
Validation binary_cross_entropy = 0.528566
Epoch 81
Validation binary_cross_entropy = 0.494160
Epoch 82
Validation binary_cross_entropy = 0.389119
Epoch 83
Validation binary_cross_entropy = 0.386911
Epoch 84
Loss = 4.2573e-01, PNorm = 64.4444, GNorm = 1.8711, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 0.626095
Epoch 85
Validation binary_cross_entropy = 0.357479
Epoch 86
Validation binary_cross_entropy = 0.468645
Epoch 87
Validation binary_cross_entropy = 0.396168
Epoch 88
Validation binary_cross_entropy = 0.541328
Epoch 89
Loss = 5.1210e-01, PNorm = 64.5409, GNorm = 2.1628, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 0.432208
Epoch 90
Validation binary_cross_entropy = 0.350868
Epoch 91
Validation binary_cross_entropy = 0.405249
Epoch 92
Validation binary_cross_entropy = 0.386058
Epoch 93
Validation binary_cross_entropy = 0.354566
Epoch 94
Loss = 4.3531e-01, PNorm = 64.6587, GNorm = 1.8769, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 0.406523
Epoch 95
Validation binary_cross_entropy = 0.481656
Epoch 96
Validation binary_cross_entropy = 0.307152
Epoch 97
Validation binary_cross_entropy = 0.415882
Epoch 98
Validation binary_cross_entropy = 0.470450
Epoch 99
Loss = 3.9036e-01, PNorm = 64.7764, GNorm = 1.3740, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.388397
Epoch 100
Validation binary_cross_entropy = 0.388972
Epoch 101
Validation binary_cross_entropy = 0.426312
Epoch 102
Validation binary_cross_entropy = 0.335682
Epoch 103
Validation binary_cross_entropy = 0.362589
Epoch 104
Loss = 4.7210e-01, PNorm = 64.9009, GNorm = 2.5020, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.367900
Epoch 105
Validation binary_cross_entropy = 0.415055
Epoch 106
Validation binary_cross_entropy = 0.353359
Epoch 107
Validation binary_cross_entropy = 0.378082
Epoch 108
Validation binary_cross_entropy = 0.394396
Epoch 109
Loss = 4.9544e-01, PNorm = 65.0290, GNorm = 4.3292, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.327714
Epoch 110
Validation binary_cross_entropy = 0.533865
Epoch 111
Validation binary_cross_entropy = 0.397293
Epoch 112
Validation binary_cross_entropy = 0.339235
Epoch 113
Validation binary_cross_entropy = 0.558443
Epoch 114
Loss = 5.1693e-01, PNorm = 65.1670, GNorm = 2.9845, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.440673
Epoch 115
Validation binary_cross_entropy = 0.307551
Epoch 116
Validation binary_cross_entropy = 0.482742
Epoch 117
Validation binary_cross_entropy = 0.441121
Epoch 118
Validation binary_cross_entropy = 0.349394
Epoch 119
Loss = 3.4035e-01, PNorm = 65.3126, GNorm = 1.3693, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.366206
Epoch 120
Validation binary_cross_entropy = 0.379846
Epoch 121
Validation binary_cross_entropy = 0.405693
Epoch 122
Validation binary_cross_entropy = 0.413864
Epoch 123
Validation binary_cross_entropy = 0.425533
Epoch 124
Loss = 3.2348e-01, PNorm = 65.4496, GNorm = 2.9674, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.547722
Epoch 125
Validation binary_cross_entropy = 0.355873
Epoch 126
Validation binary_cross_entropy = 0.392602
Epoch 127
Validation binary_cross_entropy = 0.661419
Epoch 128
Validation binary_cross_entropy = 0.368440
Epoch 129
Loss = 3.6698e-01, PNorm = 65.5949, GNorm = 1.4752, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.352750
Epoch 130
Validation binary_cross_entropy = 0.411927
Epoch 131
Validation binary_cross_entropy = 0.481080
Epoch 132
Validation binary_cross_entropy = 0.402232
Epoch 133
Validation binary_cross_entropy = 0.402027
Epoch 134
Loss = 3.8293e-01, PNorm = 65.7560, GNorm = 2.7578, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.491551
Epoch 135
Validation binary_cross_entropy = 0.410273
Epoch 136
Validation binary_cross_entropy = 0.397548
Epoch 137
Validation binary_cross_entropy = 0.445451
Epoch 138
Validation binary_cross_entropy = 0.508258
Epoch 139
Loss = 2.0537e-01, PNorm = 65.8864, GNorm = 1.0507, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.481198
Epoch 140
Validation binary_cross_entropy = 0.479820
Epoch 141
Validation binary_cross_entropy = 0.526502
Epoch 142
Validation binary_cross_entropy = 0.574982
Epoch 143
Validation binary_cross_entropy = 0.612215
Epoch 144
Loss = 1.4681e-01, PNorm = 66.0114, GNorm = 2.1856, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.639077
Epoch 145
Validation binary_cross_entropy = 0.698407
Epoch 146
Validation binary_cross_entropy = 0.632611
Epoch 147
Validation binary_cross_entropy = 0.594927
Epoch 148
Validation binary_cross_entropy = 0.561614
Epoch 149
Loss = 2.0230e-01, PNorm = 66.1266, GNorm = 2.8641, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.585597
Epoch 150
Validation binary_cross_entropy = 0.540583
Epoch 151
Validation binary_cross_entropy = 0.558106
Epoch 152
Validation binary_cross_entropy = 0.586040
Epoch 153
Validation binary_cross_entropy = 0.586533
Epoch 154
Loss = 1.6963e-01, PNorm = 66.2396, GNorm = 1.3229, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.604818
Epoch 155
Validation binary_cross_entropy = 0.590216
Epoch 156
Validation binary_cross_entropy = 0.615600
Epoch 157
Validation binary_cross_entropy = 0.632788
Epoch 158
Validation binary_cross_entropy = 0.650601
Epoch 159
Loss = 2.1232e-01, PNorm = 66.3510, GNorm = 1.5713, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.663122
Epoch 160
Validation binary_cross_entropy = 0.651727
Epoch 161
Validation binary_cross_entropy = 0.442205
Epoch 162
Validation binary_cross_entropy = 0.394072
Epoch 163
Validation binary_cross_entropy = 0.399031
Epoch 164
Loss = 2.4766e-01, PNorm = 66.4620, GNorm = 1.4658, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.440432
Epoch 165
Validation binary_cross_entropy = 0.502177
Epoch 166
Validation binary_cross_entropy = 0.531155
Epoch 167
Validation binary_cross_entropy = 0.539069
Epoch 168
Validation binary_cross_entropy = 0.534165
Epoch 169
Loss = 1.5560e-01, PNorm = 66.5774, GNorm = 1.6666, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.524106
Epoch 170
Validation binary_cross_entropy = 0.523989
Epoch 171
Validation binary_cross_entropy = 0.541298
Epoch 172
Validation binary_cross_entropy = 0.577537
Epoch 173
Validation binary_cross_entropy = 0.620675
Epoch 174
Loss = 1.9123e-01, PNorm = 66.6709, GNorm = 1.7372, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 0.647582
Epoch 175
Validation binary_cross_entropy = 0.675860
Epoch 176
Validation binary_cross_entropy = 0.667419
Epoch 177
Validation binary_cross_entropy = 0.670002
Epoch 178
Validation binary_cross_entropy = 0.691040
Epoch 179
Loss = 1.5681e-01, PNorm = 66.7594, GNorm = 6.3627, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 0.679647
Epoch 180
Validation binary_cross_entropy = 0.678242
Epoch 181
Validation binary_cross_entropy = 0.640047
Epoch 182
Validation binary_cross_entropy = 0.621404
Epoch 183
Validation binary_cross_entropy = 0.580613
Epoch 184
Loss = 9.1329e-02, PNorm = 66.8666, GNorm = 2.6225, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 0.620638
Epoch 185
Validation binary_cross_entropy = 0.618826
Epoch 186
Validation binary_cross_entropy = 0.665282
Epoch 187
Validation binary_cross_entropy = 0.700099
Epoch 188
Validation binary_cross_entropy = 0.702531
Epoch 189
Loss = 6.4512e-02, PNorm = 66.9703, GNorm = 1.2093, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 0.732174
Epoch 190
Validation binary_cross_entropy = 0.732423
Epoch 191
Validation binary_cross_entropy = 0.754431
Epoch 192
Validation binary_cross_entropy = 0.707307
Epoch 193
Validation binary_cross_entropy = 0.689016
Epoch 194
Loss = 1.0651e-01, PNorm = 67.0517, GNorm = 1.0455, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 0.663620
Epoch 195
Validation binary_cross_entropy = 0.643563
Epoch 196
Validation binary_cross_entropy = 0.656336
Epoch 197
Validation binary_cross_entropy = 0.659149
Epoch 198
Validation binary_cross_entropy = 0.628559
Epoch 199
Loss = 7.2480e-02, PNorm = 67.1227, GNorm = 0.9048, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 0.639900
Epoch 200
Validation binary_cross_entropy = 0.670813
Epoch 201
Validation binary_cross_entropy = 0.733165
Epoch 202
Validation binary_cross_entropy = 0.720059
Epoch 203
Validation binary_cross_entropy = 0.740153
Epoch 204
Loss = 3.7456e-02, PNorm = 67.2031, GNorm = 0.2152, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 0.755571
Epoch 205
Validation binary_cross_entropy = 0.781123
Epoch 206
Validation binary_cross_entropy = 0.825172
Epoch 207
Validation binary_cross_entropy = 0.877504
Epoch 208
Validation binary_cross_entropy = 0.922026
Epoch 209
Loss = 1.3644e-01, PNorm = 67.2714, GNorm = 1.5548, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 0.934244
Epoch 210
Validation binary_cross_entropy = 0.936847
Epoch 211
Validation binary_cross_entropy = 0.930200
Epoch 212
Validation binary_cross_entropy = 0.921687
Epoch 213
Validation binary_cross_entropy = 0.903498
Epoch 214
Loss = 1.7617e-01, PNorm = 67.3364, GNorm = 3.6271, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 0.873738
Epoch 215
Validation binary_cross_entropy = 0.878439
Epoch 216
Validation binary_cross_entropy = 0.836911
Epoch 217
Validation binary_cross_entropy = 0.818574
Epoch 218
Validation binary_cross_entropy = 0.836896
Epoch 219
Loss = 8.2579e-02, PNorm = 67.4039, GNorm = 1.1818, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 0.870595
Epoch 220
Validation binary_cross_entropy = 0.914395
Epoch 221
Validation binary_cross_entropy = 0.913149
Epoch 222
Validation binary_cross_entropy = 0.913756
Epoch 223
Validation binary_cross_entropy = 0.924241
Epoch 224
Loss = 6.3227e-02, PNorm = 67.4713, GNorm = 3.7481, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 0.960573
Epoch 225
Validation binary_cross_entropy = 0.996536
Epoch 226
Validation binary_cross_entropy = 1.012333
Epoch 227
Validation binary_cross_entropy = 1.016157
Epoch 228
Validation binary_cross_entropy = 1.003622
Epoch 229
Loss = 1.0721e-01, PNorm = 67.5428, GNorm = 0.8506, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 1.035002
Epoch 230
Validation binary_cross_entropy = 0.956165
Epoch 231
Validation binary_cross_entropy = 0.962658
Epoch 232
Validation binary_cross_entropy = 0.968113
Epoch 233
Validation binary_cross_entropy = 0.938806
Epoch 234
Loss = 1.6827e-01, PNorm = 67.6229, GNorm = 2.1261, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 0.975307
Epoch 235
Validation binary_cross_entropy = 0.805311
Epoch 236
Validation binary_cross_entropy = 0.753996
Epoch 237
Validation binary_cross_entropy = 0.724742
Epoch 238
Validation binary_cross_entropy = 0.699178
Epoch 239
Loss = 1.7107e-01, PNorm = 67.7074, GNorm = 2.1629, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 0.739800
Epoch 240
Validation binary_cross_entropy = 0.789324
Epoch 241
Validation binary_cross_entropy = 0.750819
Epoch 242
Validation binary_cross_entropy = 0.738809
Epoch 243
Validation binary_cross_entropy = 0.745816
Epoch 244
Loss = 1.6517e-01, PNorm = 67.8062, GNorm = 1.3805, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 0.747619
Epoch 245
Validation binary_cross_entropy = 0.760351
Epoch 246
Validation binary_cross_entropy = 0.795803
Epoch 247
Validation binary_cross_entropy = 0.843399
Epoch 248
Validation binary_cross_entropy = 0.902029
Epoch 249
Loss = 1.5845e-01, PNorm = 67.8743, GNorm = 5.7840, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 0.885994
Epoch 250
Validation binary_cross_entropy = 0.837175
Epoch 251
Validation binary_cross_entropy = 0.834124
Epoch 252
Validation binary_cross_entropy = 0.863985
Epoch 253
Validation binary_cross_entropy = 0.918563
Epoch 254
Loss = 1.6137e-01, PNorm = 67.9331, GNorm = 1.0385, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 0.823608
Epoch 255
Validation binary_cross_entropy = 0.778850
Epoch 256
Validation binary_cross_entropy = 0.781743
Epoch 257
Validation binary_cross_entropy = 0.801988
Epoch 258
Validation binary_cross_entropy = 0.872942
Epoch 259
Loss = 1.3978e-01, PNorm = 67.9915, GNorm = 2.4525, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 0.981546
Epoch 260
Validation binary_cross_entropy = 1.006060
Epoch 261
Validation binary_cross_entropy = 1.036832
Epoch 262
Validation binary_cross_entropy = 1.081043
Epoch 263
Validation binary_cross_entropy = 1.143172
Epoch 264
Loss = 5.1914e-02, PNorm = 68.0605, GNorm = 1.4255, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 1.174832
Epoch 265
Validation binary_cross_entropy = 1.150185
Epoch 266
Validation binary_cross_entropy = 1.142175
Epoch 267
Validation binary_cross_entropy = 1.163554
Epoch 268
Validation binary_cross_entropy = 1.149454
Epoch 269
Loss = 3.8407e-02, PNorm = 68.1108, GNorm = 0.9273, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 1.114847
Epoch 270
Validation binary_cross_entropy = 1.091699
Epoch 271
Validation binary_cross_entropy = 1.082084
Epoch 272
Validation binary_cross_entropy = 1.086115
Epoch 273
Validation binary_cross_entropy = 1.058876
Epoch 274
Loss = 4.1368e-02, PNorm = 68.1609, GNorm = 0.8233, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 1.024378
Epoch 275
Validation binary_cross_entropy = 1.022900
Epoch 276
Validation binary_cross_entropy = 1.059739
Epoch 277
Validation binary_cross_entropy = 1.141862
Epoch 278
Validation binary_cross_entropy = 1.157985
Epoch 279
Loss = 4.0875e-02, PNorm = 68.2106, GNorm = 0.2399, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 1.032394
Epoch 280
Validation binary_cross_entropy = 1.007251
Epoch 281
Validation binary_cross_entropy = 1.044358
Epoch 282
Validation binary_cross_entropy = 1.103137
Epoch 283
Validation binary_cross_entropy = 1.031992
Epoch 284
Loss = 9.1577e-02, PNorm = 68.2593, GNorm = 0.5152, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 0.996821
Epoch 285
Validation binary_cross_entropy = 1.025843
Epoch 286
Validation binary_cross_entropy = 1.102168
Epoch 287
Validation binary_cross_entropy = 1.088647
Epoch 288
Validation binary_cross_entropy = 1.096987
Epoch 289
Loss = 6.6537e-02, PNorm = 68.3209, GNorm = 1.1367, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 1.127356
Epoch 290
Validation binary_cross_entropy = 1.191445
Epoch 291
Validation binary_cross_entropy = 1.210304
Epoch 292
Validation binary_cross_entropy = 1.259330
Epoch 293
Validation binary_cross_entropy = 1.323222
Epoch 294
Loss = 5.5544e-02, PNorm = 68.3817, GNorm = 1.4752, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 1.327587
Epoch 295
Validation binary_cross_entropy = 1.307706
Epoch 296
Validation binary_cross_entropy = 1.311044
Epoch 297
Validation binary_cross_entropy = 1.358216
Epoch 298
Validation binary_cross_entropy = 1.458270
Epoch 299
Loss = 9.2846e-02, PNorm = 68.4349, GNorm = 1.0045, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 1.349580
Model 0 best validation binary_cross_entropy = 0.288288 on epoch 7
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.244754
Ensemble test binary_cross_entropy = 0.244754
Fold 4
Splitting data with seed 4
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.4, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=800, bias=False)
        (W_h): Linear(in_features=800, out_features=800, bias=False)
        (W_o): Linear(in_features=933, out_features=800, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.4, inplace=False)
    (1): Linear(in_features=3100, out_features=800, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.4, inplace=False)
    (4): Linear(in_features=800, out_features=800, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.4, inplace=False)
    (7): Linear(in_features=800, out_features=1, bias=True)
  )
)
Number of parameters = 4,627,201
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 0.288492
Epoch 1
Validation binary_cross_entropy = 0.299842
Epoch 2
Validation binary_cross_entropy = 0.543299
Epoch 3
Validation binary_cross_entropy = 0.315291
Epoch 4
Loss = 8.1265e-01, PNorm = 63.1515, GNorm = 7.8907, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 0.296001
Epoch 5
Validation binary_cross_entropy = 0.422690
Epoch 6
Validation binary_cross_entropy = 0.719525
Epoch 7
Validation binary_cross_entropy = 0.306277
Epoch 8
Validation binary_cross_entropy = 0.396395
Epoch 9
Loss = 6.9810e-01, PNorm = 63.2896, GNorm = 6.6396, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 0.909441
Epoch 10
Validation binary_cross_entropy = 0.319208
Epoch 11
Validation binary_cross_entropy = 0.344195
Epoch 12
Validation binary_cross_entropy = 0.870440
Epoch 13
Validation binary_cross_entropy = 0.701116
Epoch 14
Loss = 8.1211e-01, PNorm = 63.4412, GNorm = 4.5585, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 0.306238
Epoch 15
Validation binary_cross_entropy = 0.491329
Epoch 16
Validation binary_cross_entropy = 0.811347
Epoch 17
Validation binary_cross_entropy = 0.366089
Epoch 18
Validation binary_cross_entropy = 0.339635
Epoch 19
Loss = 7.1207e-01, PNorm = 63.5675, GNorm = 3.5811, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 0.571972
Epoch 20
Validation binary_cross_entropy = 0.852517
Epoch 21
Validation binary_cross_entropy = 0.555954
Epoch 22
Validation binary_cross_entropy = 0.412846
Epoch 23
Validation binary_cross_entropy = 0.692608
Epoch 24
Loss = 6.7237e-01, PNorm = 63.6636, GNorm = 3.9711, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 0.614873
Epoch 25
Validation binary_cross_entropy = 0.441450
Epoch 26
Validation binary_cross_entropy = 0.397073
Epoch 27
Validation binary_cross_entropy = 0.483526
Epoch 28
Validation binary_cross_entropy = 0.552853
Epoch 29
Loss = 5.9715e-01, PNorm = 63.7372, GNorm = 2.5620, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 0.477576
Epoch 30
Validation binary_cross_entropy = 0.450970
Epoch 31
Validation binary_cross_entropy = 0.475460
Epoch 32
Validation binary_cross_entropy = 0.556908
Epoch 33
Validation binary_cross_entropy = 0.406190
Epoch 34
Loss = 6.0678e-01, PNorm = 63.8067, GNorm = 2.5643, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 0.381212
Epoch 35
Validation binary_cross_entropy = 0.412417
Epoch 36
Validation binary_cross_entropy = 0.497742
Epoch 37
Validation binary_cross_entropy = 0.578227
Epoch 38
Validation binary_cross_entropy = 0.503798
Epoch 39
Loss = 7.0062e-01, PNorm = 63.8593, GNorm = 3.6762, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 0.410573
Epoch 40
Validation binary_cross_entropy = 0.462313
Epoch 41
Validation binary_cross_entropy = 0.612047
Epoch 42
Validation binary_cross_entropy = 0.558900
Epoch 43
Validation binary_cross_entropy = 0.401490
Epoch 44
Loss = 5.9761e-01, PNorm = 63.9216, GNorm = 2.6747, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 0.397668
Epoch 45
Validation binary_cross_entropy = 0.490424
Epoch 46
Validation binary_cross_entropy = 0.620691
Epoch 47
Validation binary_cross_entropy = 0.606092
Epoch 48
Validation binary_cross_entropy = 0.553369
Epoch 49
Loss = 5.9209e-01, PNorm = 63.9908, GNorm = 1.9154, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 0.463137
Epoch 50
Validation binary_cross_entropy = 0.449482
Epoch 51
Validation binary_cross_entropy = 0.484529
Epoch 52
Validation binary_cross_entropy = 0.498411
Epoch 53
Validation binary_cross_entropy = 0.439562
Epoch 54
Loss = 5.6853e-01, PNorm = 64.0589, GNorm = 1.9765, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 0.428338
Epoch 55
Validation binary_cross_entropy = 0.488247
Epoch 56
Validation binary_cross_entropy = 0.557612
Epoch 57
Validation binary_cross_entropy = 0.533687
Epoch 58
Validation binary_cross_entropy = 0.515251
Epoch 59
Loss = 6.5696e-01, PNorm = 64.1299, GNorm = 2.6334, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 0.419050
Epoch 60
Validation binary_cross_entropy = 0.417131
Epoch 61
Validation binary_cross_entropy = 0.467819
Epoch 62
Validation binary_cross_entropy = 0.473204
Epoch 63
Validation binary_cross_entropy = 0.488440
Epoch 64
Loss = 5.2813e-01, PNorm = 64.2105, GNorm = 1.6724, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 0.547683
Epoch 65
Validation binary_cross_entropy = 0.401022
Epoch 66
Validation binary_cross_entropy = 0.297535
Epoch 67
Validation binary_cross_entropy = 0.409309
Epoch 68
Validation binary_cross_entropy = 0.669960
Epoch 69
Loss = 5.9968e-01, PNorm = 64.2814, GNorm = 2.0931, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 0.589756
Epoch 70
Validation binary_cross_entropy = 0.451441
Epoch 71
Validation binary_cross_entropy = 0.416110
Epoch 72
Validation binary_cross_entropy = 0.482423
Epoch 73
Validation binary_cross_entropy = 0.553170
Epoch 74
Loss = 5.6295e-01, PNorm = 64.3738, GNorm = 1.3523, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 0.431929
Epoch 75
Validation binary_cross_entropy = 0.379163
Epoch 76
Validation binary_cross_entropy = 0.405960
Epoch 77
Validation binary_cross_entropy = 0.412164
Epoch 78
Validation binary_cross_entropy = 0.447813
Epoch 79
Loss = 5.2126e-01, PNorm = 64.4663, GNorm = 2.0992, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 0.393825
Epoch 80
Validation binary_cross_entropy = 0.369460
Epoch 81
Validation binary_cross_entropy = 0.486076
Epoch 82
Validation binary_cross_entropy = 0.467802
Epoch 83
Validation binary_cross_entropy = 0.370557
Epoch 84
Loss = 4.3767e-01, PNorm = 64.5614, GNorm = 1.5679, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 0.472103
Epoch 85
Validation binary_cross_entropy = 0.459200
Epoch 86
Validation binary_cross_entropy = 0.339749
Epoch 87
Validation binary_cross_entropy = 0.542451
Epoch 88
Validation binary_cross_entropy = 0.441087
Epoch 89
Loss = 4.1964e-01, PNorm = 64.6586, GNorm = 2.0495, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 0.319750
Epoch 90
Validation binary_cross_entropy = 0.373408
Epoch 91
Validation binary_cross_entropy = 0.502973
Epoch 92
Validation binary_cross_entropy = 0.405448
Epoch 93
Validation binary_cross_entropy = 0.317231
Epoch 94
Loss = 4.8523e-01, PNorm = 64.7657, GNorm = 1.4608, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 0.420130
Epoch 95
Validation binary_cross_entropy = 0.429675
Epoch 96
Validation binary_cross_entropy = 0.319749
Epoch 97
Validation binary_cross_entropy = 0.380187
Epoch 98
Validation binary_cross_entropy = 0.307637
Epoch 99
Loss = 3.4253e-01, PNorm = 64.8643, GNorm = 2.1386, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.466270
Epoch 100
Validation binary_cross_entropy = 0.345845
Epoch 101
Validation binary_cross_entropy = 0.288632
Epoch 102
Validation binary_cross_entropy = 0.473014
Epoch 103
Validation binary_cross_entropy = 0.397956
Epoch 104
Loss = 3.2407e-01, PNorm = 64.9747, GNorm = 1.5386, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.344894
Epoch 105
Validation binary_cross_entropy = 0.539123
Epoch 106
Validation binary_cross_entropy = 0.433450
Epoch 107
Validation binary_cross_entropy = 0.368778
Epoch 108
Validation binary_cross_entropy = 0.463620
Epoch 109
Loss = 3.0781e-01, PNorm = 65.1021, GNorm = 2.7088, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.372240
Epoch 110
Validation binary_cross_entropy = 0.395180
Epoch 111
Validation binary_cross_entropy = 0.418839
Epoch 112
Validation binary_cross_entropy = 0.319872
Epoch 113
Validation binary_cross_entropy = 0.435981
Epoch 114
Loss = 3.5858e-01, PNorm = 65.2264, GNorm = 3.6772, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.358717
Epoch 115
Validation binary_cross_entropy = 0.303516
Epoch 116
Validation binary_cross_entropy = 0.335892
Epoch 117
Validation binary_cross_entropy = 0.413988
Epoch 118
Validation binary_cross_entropy = 0.425450
Epoch 119
Loss = 4.0860e-01, PNorm = 65.3553, GNorm = 2.6665, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.348972
Epoch 120
Validation binary_cross_entropy = 0.601076
Epoch 121
Validation binary_cross_entropy = 0.320109
Epoch 122
Validation binary_cross_entropy = 0.287189
Epoch 123
Validation binary_cross_entropy = 0.365995
Epoch 124
Loss = 3.1365e-01, PNorm = 65.4845, GNorm = 1.5758, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.553133
Epoch 125
Validation binary_cross_entropy = 0.504318
Epoch 126
Validation binary_cross_entropy = 0.347122
Epoch 127
Validation binary_cross_entropy = 0.374508
Epoch 128
Validation binary_cross_entropy = 0.486244
Epoch 129
Loss = 2.6749e-01, PNorm = 65.6170, GNorm = 2.7061, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.451229
Epoch 130
Validation binary_cross_entropy = 0.361152
Epoch 131
Validation binary_cross_entropy = 0.363787
Epoch 132
Validation binary_cross_entropy = 0.408224
Epoch 133
Validation binary_cross_entropy = 0.421603
Epoch 134
Loss = 2.5335e-01, PNorm = 65.7448, GNorm = 1.2757, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.411673
Epoch 135
Validation binary_cross_entropy = 0.432730
Epoch 136
Validation binary_cross_entropy = 0.468344
Epoch 137
Validation binary_cross_entropy = 0.480847
Epoch 138
Validation binary_cross_entropy = 0.549372
Epoch 139
Loss = 3.0719e-01, PNorm = 65.8600, GNorm = 3.6005, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.472428
Epoch 140
Validation binary_cross_entropy = 0.436960
Epoch 141
Validation binary_cross_entropy = 0.422572
Epoch 142
Validation binary_cross_entropy = 0.559141
Epoch 143
Validation binary_cross_entropy = 0.476101
Epoch 144
Loss = 1.5877e-01, PNorm = 65.9867, GNorm = 0.8794, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.457860
Epoch 145
Validation binary_cross_entropy = 0.493195
Epoch 146
Validation binary_cross_entropy = 0.514537
Epoch 147
Validation binary_cross_entropy = 0.564307
Epoch 148
Validation binary_cross_entropy = 0.577007
Epoch 149
Loss = 1.8379e-01, PNorm = 66.1100, GNorm = 4.5137, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.597148
Epoch 150
Validation binary_cross_entropy = 0.613608
Epoch 151
Validation binary_cross_entropy = 0.639884
Epoch 152
Validation binary_cross_entropy = 0.633445
Epoch 153
Validation binary_cross_entropy = 0.632491
Epoch 154
Loss = 1.1833e-01, PNorm = 66.2233, GNorm = 1.8283, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.649407
Epoch 155
Validation binary_cross_entropy = 0.676634
Epoch 156
Validation binary_cross_entropy = 0.710803
Epoch 157
Validation binary_cross_entropy = 0.708321
Epoch 158
Validation binary_cross_entropy = 0.680733
Epoch 159
Loss = 1.4231e-01, PNorm = 66.3165, GNorm = 4.4290, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.670964
Epoch 160
Validation binary_cross_entropy = 0.645692
Epoch 161
Validation binary_cross_entropy = 0.615526
Epoch 162
Validation binary_cross_entropy = 0.563780
Epoch 163
Validation binary_cross_entropy = 0.553024
Epoch 164
Loss = 2.4194e-01, PNorm = 66.4023, GNorm = 1.5600, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.494587
Epoch 165
Validation binary_cross_entropy = 0.501754
Epoch 166
Validation binary_cross_entropy = 0.497926
Epoch 167
Validation binary_cross_entropy = 0.507897
Epoch 168
Validation binary_cross_entropy = 0.542008
Epoch 169
Loss = 2.2428e-01, PNorm = 66.5016, GNorm = 1.3830, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.571778
Epoch 170
Validation binary_cross_entropy = 0.660411
Epoch 171
Validation binary_cross_entropy = 0.584957
Epoch 172
Validation binary_cross_entropy = 0.597132
Epoch 173
Validation binary_cross_entropy = 0.550206
Epoch 174
Loss = 1.2431e-01, PNorm = 66.5879, GNorm = 0.7695, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 0.601356
Epoch 175
Validation binary_cross_entropy = 0.627331
Epoch 176
Validation binary_cross_entropy = 0.572003
Epoch 177
Validation binary_cross_entropy = 0.595815
Epoch 178
Validation binary_cross_entropy = 0.602376
Epoch 179
Loss = 1.7807e-01, PNorm = 66.6700, GNorm = 1.1253, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 0.616359
Epoch 180
Validation binary_cross_entropy = 0.691526
Epoch 181
Validation binary_cross_entropy = 0.667546
Epoch 182
Validation binary_cross_entropy = 0.670233
Epoch 183
Validation binary_cross_entropy = 0.708248
Epoch 184
Loss = 1.2918e-01, PNorm = 66.7524, GNorm = 1.1723, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 0.763388
Epoch 185
Validation binary_cross_entropy = 0.787253
Epoch 186
Validation binary_cross_entropy = 0.796164
Epoch 187
Validation binary_cross_entropy = 0.846020
Epoch 188
Validation binary_cross_entropy = 0.888438
Epoch 189
Loss = 8.1052e-02, PNorm = 66.8272, GNorm = 1.2412, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 0.893048
Epoch 190
Validation binary_cross_entropy = 0.866334
Epoch 191
Validation binary_cross_entropy = 0.881472
Epoch 192
Validation binary_cross_entropy = 0.844109
Epoch 193
Validation binary_cross_entropy = 0.823337
Epoch 194
Loss = 1.7810e-01, PNorm = 66.8872, GNorm = 1.7229, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 0.738904
Epoch 195
Validation binary_cross_entropy = 0.698282
Epoch 196
Validation binary_cross_entropy = 0.697426
Epoch 197
Validation binary_cross_entropy = 0.688727
Epoch 198
Validation binary_cross_entropy = 0.647209
Epoch 199
Loss = 1.2401e-01, PNorm = 66.9585, GNorm = 1.4344, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 0.673292
Epoch 200
Validation binary_cross_entropy = 0.736361
Epoch 201
Validation binary_cross_entropy = 0.833612
Epoch 202
Validation binary_cross_entropy = 0.830946
Epoch 203
Validation binary_cross_entropy = 0.786475
Epoch 204
Loss = 6.5641e-02, PNorm = 67.0398, GNorm = 3.3544, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 0.812134
Epoch 205
Validation binary_cross_entropy = 0.881436
Epoch 206
Validation binary_cross_entropy = 0.866491
Epoch 207
Validation binary_cross_entropy = 0.842056
Epoch 208
Validation binary_cross_entropy = 0.800843
Epoch 209
Loss = 7.4235e-02, PNorm = 67.1174, GNorm = 1.1437, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 0.774655
Epoch 210
Validation binary_cross_entropy = 0.765011
Epoch 211
Validation binary_cross_entropy = 0.772774
Epoch 212
Validation binary_cross_entropy = 0.788674
Epoch 213
Validation binary_cross_entropy = 0.818022
Epoch 214
Loss = 1.6156e-01, PNorm = 67.1897, GNorm = 1.0422, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 0.860095
Epoch 215
Validation binary_cross_entropy = 0.887749
Epoch 216
Validation binary_cross_entropy = 0.879551
Epoch 217
Validation binary_cross_entropy = 0.952249
Epoch 218
Validation binary_cross_entropy = 0.940468
Epoch 219
Loss = 9.2942e-02, PNorm = 67.2484, GNorm = 2.2695, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 0.939447
Epoch 220
Validation binary_cross_entropy = 0.929503
Epoch 221
Validation binary_cross_entropy = 0.941944
Epoch 222
Validation binary_cross_entropy = 0.956086
Epoch 223
Validation binary_cross_entropy = 0.788120
Epoch 224
Loss = 5.3579e-02, PNorm = 67.3229, GNorm = 1.1557, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 0.711368
Epoch 225
Validation binary_cross_entropy = 0.696573
Epoch 226
Validation binary_cross_entropy = 0.707411
Epoch 227
Validation binary_cross_entropy = 0.650048
Epoch 228
Validation binary_cross_entropy = 0.618063
Epoch 229
Loss = 1.3217e-01, PNorm = 67.4078, GNorm = 1.2706, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 0.646415
Epoch 230
Validation binary_cross_entropy = 0.704910
Epoch 231
Validation binary_cross_entropy = 0.695859
Epoch 232
Validation binary_cross_entropy = 0.696799
Epoch 233
Validation binary_cross_entropy = 0.776171
Epoch 234
Loss = 5.0509e-02, PNorm = 67.4974, GNorm = 0.9569, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 0.886606
Epoch 235
Validation binary_cross_entropy = 0.952678
Epoch 236
Validation binary_cross_entropy = 0.862890
Epoch 237
Validation binary_cross_entropy = 0.851269
Epoch 238
Validation binary_cross_entropy = 0.834280
Epoch 239
Loss = 4.9108e-02, PNorm = 67.5672, GNorm = 0.2768, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 0.807954
Epoch 240
Validation binary_cross_entropy = 0.786730
Epoch 241
Validation binary_cross_entropy = 0.795628
Epoch 242
Validation binary_cross_entropy = 0.818009
Epoch 243
Validation binary_cross_entropy = 0.827724
Epoch 244
Loss = 5.5007e-02, PNorm = 67.6227, GNorm = 0.4361, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 0.828937
Epoch 245
Validation binary_cross_entropy = 0.841685
Epoch 246
Validation binary_cross_entropy = 0.915851
Epoch 247
Validation binary_cross_entropy = 0.939810
Epoch 248
Validation binary_cross_entropy = 0.875095
Epoch 249
Loss = 4.5072e-02, PNorm = 67.6748, GNorm = 2.0576, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 0.878501
Epoch 250
Validation binary_cross_entropy = 0.980757
Epoch 251
Validation binary_cross_entropy = 1.048447
Epoch 252
Validation binary_cross_entropy = 0.983834
Epoch 253
Validation binary_cross_entropy = 0.985679
Epoch 254
Loss = 4.4396e-02, PNorm = 67.7323, GNorm = 0.2994, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 1.050910
Epoch 255
Validation binary_cross_entropy = 1.091812
Epoch 256
Validation binary_cross_entropy = 1.088791
Epoch 257
Validation binary_cross_entropy = 1.028713
Epoch 258
Validation binary_cross_entropy = 1.025065
Epoch 259
Loss = 3.3006e-02, PNorm = 67.7895, GNorm = 0.7796, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 1.063460
Epoch 260
Validation binary_cross_entropy = 1.131845
Epoch 261
Validation binary_cross_entropy = 1.161702
Epoch 262
Validation binary_cross_entropy = 1.119502
Epoch 263
Validation binary_cross_entropy = 1.064125
Epoch 264
Loss = 1.2071e-01, PNorm = 67.8354, GNorm = 0.3764, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 0.972817
Epoch 265
Validation binary_cross_entropy = 0.998623
Epoch 266
Validation binary_cross_entropy = 1.004773
Epoch 267
Validation binary_cross_entropy = 0.964099
Epoch 268
Validation binary_cross_entropy = 0.970218
Epoch 269
Loss = 6.6356e-02, PNorm = 67.8870, GNorm = 2.9820, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 1.001966
Epoch 270
Validation binary_cross_entropy = 0.949548
Epoch 271
Validation binary_cross_entropy = 0.882026
Epoch 272
Validation binary_cross_entropy = 0.850530
Epoch 273
Validation binary_cross_entropy = 0.849377
Epoch 274
Loss = 2.1530e-02, PNorm = 67.9460, GNorm = 0.7333, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 0.859311
Epoch 275
Validation binary_cross_entropy = 0.897594
Epoch 276
Validation binary_cross_entropy = 0.988357
Epoch 277
Validation binary_cross_entropy = 0.980737
Epoch 278
Validation binary_cross_entropy = 0.999562
Epoch 279
Loss = 1.1250e-02, PNorm = 67.9977, GNorm = 0.2630, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 1.050128
Epoch 280
Validation binary_cross_entropy = 1.132589
Epoch 281
Validation binary_cross_entropy = 1.350404
Epoch 282
Validation binary_cross_entropy = 1.076796
Epoch 283
Validation binary_cross_entropy = 0.924323
Epoch 284
Loss = 2.7630e-02, PNorm = 68.0418, GNorm = 1.7022, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 0.874866
Epoch 285
Validation binary_cross_entropy = 0.879988
Epoch 286
Validation binary_cross_entropy = 0.899492
Epoch 287
Validation binary_cross_entropy = 0.888202
Epoch 288
Validation binary_cross_entropy = 0.901789
Epoch 289
Loss = 8.6714e-02, PNorm = 68.0950, GNorm = 2.2805, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 0.947751
Epoch 290
Validation binary_cross_entropy = 0.997188
Epoch 291
Validation binary_cross_entropy = 1.072040
Epoch 292
Validation binary_cross_entropy = 1.155267
Epoch 293
Validation binary_cross_entropy = 1.246546
Epoch 294
Loss = 8.2947e-02, PNorm = 68.1399, GNorm = 0.4094, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 1.090367
Epoch 295
Validation binary_cross_entropy = 1.105184
Epoch 296
Validation binary_cross_entropy = 1.107655
Epoch 297
Validation binary_cross_entropy = 1.196965
Epoch 298
Validation binary_cross_entropy = 1.177516
Epoch 299
Loss = 1.1430e-01, PNorm = 68.1812, GNorm = 1.5346, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 1.154961
Model 0 best validation binary_cross_entropy = 0.287189 on epoch 122
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.193636
Ensemble test binary_cross_entropy = 0.193636
Fold 5
Splitting data with seed 5
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.4, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=800, bias=False)
        (W_h): Linear(in_features=800, out_features=800, bias=False)
        (W_o): Linear(in_features=933, out_features=800, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.4, inplace=False)
    (1): Linear(in_features=3100, out_features=800, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.4, inplace=False)
    (4): Linear(in_features=800, out_features=800, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.4, inplace=False)
    (7): Linear(in_features=800, out_features=1, bias=True)
  )
)
Number of parameters = 4,627,201
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 0.289534
Epoch 1
Validation binary_cross_entropy = 0.303106
Epoch 2
Validation binary_cross_entropy = 0.591807
Epoch 3
Validation binary_cross_entropy = 0.330373
Epoch 4
Loss = 7.9908e-01, PNorm = 63.1519, GNorm = 7.7105, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 0.296419
Epoch 5
Validation binary_cross_entropy = 0.581647
Epoch 6
Validation binary_cross_entropy = 0.814832
Epoch 7
Validation binary_cross_entropy = 0.313778
Epoch 8
Validation binary_cross_entropy = 0.428733
Epoch 9
Loss = 7.0348e-01, PNorm = 63.2918, GNorm = 6.8510, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 0.757956
Epoch 10
Validation binary_cross_entropy = 0.331210
Epoch 11
Validation binary_cross_entropy = 0.318045
Epoch 12
Validation binary_cross_entropy = 0.867532
Epoch 13
Validation binary_cross_entropy = 0.479021
Epoch 14
Loss = 8.2486e-01, PNorm = 63.4349, GNorm = 7.0120, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 0.308802
Epoch 15
Validation binary_cross_entropy = 0.827013
Epoch 16
Validation binary_cross_entropy = 0.475661
Epoch 17
Validation binary_cross_entropy = 0.339257
Epoch 18
Validation binary_cross_entropy = 0.470448
Epoch 19
Loss = 6.7259e-01, PNorm = 63.5626, GNorm = 3.9253, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 0.482667
Epoch 20
Validation binary_cross_entropy = 0.373375
Epoch 21
Validation binary_cross_entropy = 0.426444
Epoch 22
Validation binary_cross_entropy = 0.367177
Epoch 23
Validation binary_cross_entropy = 0.346998
Epoch 24
Loss = 6.9311e-01, PNorm = 63.6544, GNorm = 3.2835, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 0.599024
Epoch 25
Validation binary_cross_entropy = 0.702147
Epoch 26
Validation binary_cross_entropy = 0.486602
Epoch 27
Validation binary_cross_entropy = 0.420818
Epoch 28
Validation binary_cross_entropy = 0.463863
Epoch 29
Loss = 7.3363e-01, PNorm = 63.7302, GNorm = 4.3377, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 0.629639
Epoch 30
Validation binary_cross_entropy = 0.407996
Epoch 31
Validation binary_cross_entropy = 0.377826
Epoch 32
Validation binary_cross_entropy = 0.461973
Epoch 33
Validation binary_cross_entropy = 0.694770
Epoch 34
Loss = 6.0167e-01, PNorm = 63.7929, GNorm = 3.0931, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 0.718450
Epoch 35
Validation binary_cross_entropy = 0.418568
Epoch 36
Validation binary_cross_entropy = 0.372510
Epoch 37
Validation binary_cross_entropy = 0.517316
Epoch 38
Validation binary_cross_entropy = 0.573922
Epoch 39
Loss = 6.2582e-01, PNorm = 63.8460, GNorm = 2.6273, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 0.506180
Epoch 40
Validation binary_cross_entropy = 0.449721
Epoch 41
Validation binary_cross_entropy = 0.456236
Epoch 42
Validation binary_cross_entropy = 0.528537
Epoch 43
Validation binary_cross_entropy = 0.576312
Epoch 44
Loss = 6.4338e-01, PNorm = 63.9045, GNorm = 2.7616, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 0.442549
Epoch 45
Validation binary_cross_entropy = 0.399084
Epoch 46
Validation binary_cross_entropy = 0.449642
Epoch 47
Validation binary_cross_entropy = 0.616498
Epoch 48
Validation binary_cross_entropy = 0.657342
Epoch 49
Loss = 5.8795e-01, PNorm = 63.9608, GNorm = 2.2239, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 0.526729
Epoch 50
Validation binary_cross_entropy = 0.453491
Epoch 51
Validation binary_cross_entropy = 0.410425
Epoch 52
Validation binary_cross_entropy = 0.480687
Epoch 53
Validation binary_cross_entropy = 0.599847
Epoch 54
Loss = 5.9766e-01, PNorm = 64.0206, GNorm = 2.2348, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 0.512638
Epoch 55
Validation binary_cross_entropy = 0.367258
Epoch 56
Validation binary_cross_entropy = 0.442367
Epoch 57
Validation binary_cross_entropy = 0.508343
Epoch 58
Validation binary_cross_entropy = 0.389574
Epoch 59
Loss = 4.5220e-01, PNorm = 64.0929, GNorm = 1.9649, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 0.609938
Epoch 60
Validation binary_cross_entropy = 0.388002
Epoch 61
Validation binary_cross_entropy = 0.379866
Epoch 62
Validation binary_cross_entropy = 0.372816
Epoch 63
Validation binary_cross_entropy = 0.473468
Epoch 64
Loss = 5.3463e-01, PNorm = 64.1623, GNorm = 2.0874, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 0.537794
Epoch 65
Validation binary_cross_entropy = 0.342271
Epoch 66
Validation binary_cross_entropy = 0.504715
Epoch 67
Validation binary_cross_entropy = 0.570594
Epoch 68
Validation binary_cross_entropy = 0.305778
Epoch 69
Loss = 5.3503e-01, PNorm = 64.2516, GNorm = 3.6953, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 0.524626
Epoch 70
Validation binary_cross_entropy = 0.718144
Epoch 71
Validation binary_cross_entropy = 0.403082
Epoch 72
Validation binary_cross_entropy = 0.344761
Epoch 73
Validation binary_cross_entropy = 0.483254
Epoch 74
Loss = 6.1048e-01, PNorm = 64.3392, GNorm = 2.8279, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 0.719330
Epoch 75
Validation binary_cross_entropy = 0.615702
Epoch 76
Validation binary_cross_entropy = 0.365460
Epoch 77
Validation binary_cross_entropy = 0.326033
Epoch 78
Validation binary_cross_entropy = 0.632319
Epoch 79
Loss = 5.5140e-01, PNorm = 64.4419, GNorm = 4.0891, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 0.348579
Epoch 80
Validation binary_cross_entropy = 0.317703
Epoch 81
Validation binary_cross_entropy = 0.589992
Epoch 82
Validation binary_cross_entropy = 0.494011
Epoch 83
Validation binary_cross_entropy = 0.358935
Epoch 84
Loss = 5.2401e-01, PNorm = 64.5326, GNorm = 2.0422, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 0.335201
Epoch 85
Validation binary_cross_entropy = 0.353149
Epoch 86
Validation binary_cross_entropy = 0.439620
Epoch 87
Validation binary_cross_entropy = 0.475514
Epoch 88
Validation binary_cross_entropy = 0.340624
Epoch 89
Loss = 4.3803e-01, PNorm = 64.6462, GNorm = 1.6727, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 0.349045
Epoch 90
Validation binary_cross_entropy = 0.380066
Epoch 91
Validation binary_cross_entropy = 0.366312
Epoch 92
Validation binary_cross_entropy = 0.343032
Epoch 93
Validation binary_cross_entropy = 0.418403
Epoch 94
Loss = 4.7908e-01, PNorm = 64.7488, GNorm = 2.7550, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 0.304596
Epoch 95
Validation binary_cross_entropy = 0.359888
Epoch 96
Validation binary_cross_entropy = 0.406973
Epoch 97
Validation binary_cross_entropy = 0.469343
Epoch 98
Validation binary_cross_entropy = 0.305717
Epoch 99
Loss = 4.1188e-01, PNorm = 64.8650, GNorm = 3.1658, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.373944
Epoch 100
Validation binary_cross_entropy = 0.339074
Epoch 101
Validation binary_cross_entropy = 0.419456
Epoch 102
Validation binary_cross_entropy = 0.391161
Epoch 103
Validation binary_cross_entropy = 0.444045
Epoch 104
Loss = 4.3906e-01, PNorm = 64.9807, GNorm = 2.7561, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.333910
Epoch 105
Validation binary_cross_entropy = 0.528069
Epoch 106
Validation binary_cross_entropy = 0.380991
Epoch 107
Validation binary_cross_entropy = 0.283916
Epoch 108
Validation binary_cross_entropy = 0.322393
Epoch 109
Loss = 3.7935e-01, PNorm = 65.0981, GNorm = 1.8180, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.461531
Epoch 110
Validation binary_cross_entropy = 0.351225
Epoch 111
Validation binary_cross_entropy = 0.306402
Epoch 112
Validation binary_cross_entropy = 0.385776
Epoch 113
Validation binary_cross_entropy = 0.452676
Epoch 114
Loss = 3.3111e-01, PNorm = 65.2166, GNorm = 1.5309, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.330124
Epoch 115
Validation binary_cross_entropy = 0.363563
Epoch 116
Validation binary_cross_entropy = 0.395418
Epoch 117
Validation binary_cross_entropy = 0.396988
Epoch 118
Validation binary_cross_entropy = 0.398406
Epoch 119
Loss = 2.2646e-01, PNorm = 65.3474, GNorm = 1.9135, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.725368
Epoch 120
Validation binary_cross_entropy = 0.425772
Epoch 121
Validation binary_cross_entropy = 0.395078
Epoch 122
Validation binary_cross_entropy = 0.446521
Epoch 123
Validation binary_cross_entropy = 0.327633
Epoch 124
Loss = 3.6488e-01, PNorm = 65.4679, GNorm = 2.2302, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.322538
Epoch 125
Validation binary_cross_entropy = 0.393764
Epoch 126
Validation binary_cross_entropy = 0.426126
Epoch 127
Validation binary_cross_entropy = 0.393221
Epoch 128
Validation binary_cross_entropy = 0.438141
Epoch 129
Loss = 2.6660e-01, PNorm = 65.6184, GNorm = 2.3785, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.558798
Epoch 130
Validation binary_cross_entropy = 0.497348
Epoch 131
Validation binary_cross_entropy = 0.737775
Epoch 132
Validation binary_cross_entropy = 0.519649
Epoch 133
Validation binary_cross_entropy = 0.451083
Epoch 134
Loss = 2.3550e-01, PNorm = 65.7456, GNorm = 1.1615, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.483131
Epoch 135
Validation binary_cross_entropy = 0.556122
Epoch 136
Validation binary_cross_entropy = 0.431436
Epoch 137
Validation binary_cross_entropy = 0.420294
Epoch 138
Validation binary_cross_entropy = 0.467153
Epoch 139
Loss = 1.8987e-01, PNorm = 65.8634, GNorm = 1.2742, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.565086
Epoch 140
Validation binary_cross_entropy = 0.539815
Epoch 141
Validation binary_cross_entropy = 0.558009
Epoch 142
Validation binary_cross_entropy = 0.595306
Epoch 143
Validation binary_cross_entropy = 0.583532
Epoch 144
Loss = 2.5914e-01, PNorm = 65.9621, GNorm = 1.3477, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.554689
Epoch 145
Validation binary_cross_entropy = 0.535855
Epoch 146
Validation binary_cross_entropy = 0.530141
Epoch 147
Validation binary_cross_entropy = 0.516437
Epoch 148
Validation binary_cross_entropy = 0.532888
Epoch 149
Loss = 2.7930e-01, PNorm = 66.0599, GNorm = 2.5731, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.547819
Epoch 150
Validation binary_cross_entropy = 0.623586
Epoch 151
Validation binary_cross_entropy = 0.617215
Epoch 152
Validation binary_cross_entropy = 0.631802
Epoch 153
Validation binary_cross_entropy = 0.614046
Epoch 154
Loss = 1.8617e-01, PNorm = 66.1565, GNorm = 1.0143, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.614184
Epoch 155
Validation binary_cross_entropy = 0.582074
Epoch 156
Validation binary_cross_entropy = 0.636387
Epoch 157
Validation binary_cross_entropy = 0.591666
Epoch 158
Validation binary_cross_entropy = 0.612196
Epoch 159
Loss = 1.4365e-01, PNorm = 66.2581, GNorm = 2.6039, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.593060
Epoch 160
Validation binary_cross_entropy = 0.617739
Epoch 161
Validation binary_cross_entropy = 0.645798
Epoch 162
Validation binary_cross_entropy = 0.712961
Epoch 163
Validation binary_cross_entropy = 0.653916
Epoch 164
Loss = 1.4867e-01, PNorm = 66.3599, GNorm = 2.2036, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.673173
Epoch 165
Validation binary_cross_entropy = 0.686101
Epoch 166
Validation binary_cross_entropy = 0.636391
Epoch 167
Validation binary_cross_entropy = 0.605493
Epoch 168
Validation binary_cross_entropy = 0.611337
Epoch 169
Loss = 1.2600e-01, PNorm = 66.4578, GNorm = 0.7406, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.634608
Epoch 170
Validation binary_cross_entropy = 0.656961
Epoch 171
Validation binary_cross_entropy = 0.705853
Epoch 172
Validation binary_cross_entropy = 0.740981
Epoch 173
Validation binary_cross_entropy = 0.811262
Epoch 174
Loss = 1.8654e-01, PNorm = 66.5507, GNorm = 0.7556, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 0.690891
Epoch 175
Validation binary_cross_entropy = 0.662704
Epoch 176
Validation binary_cross_entropy = 0.685672
Epoch 177
Validation binary_cross_entropy = 0.692484
Epoch 178
Validation binary_cross_entropy = 0.660337
Epoch 179
Loss = 1.0502e-01, PNorm = 66.6497, GNorm = 1.1772, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 0.752122
Epoch 180
Validation binary_cross_entropy = 0.722603
Epoch 181
Validation binary_cross_entropy = 0.768387
Epoch 182
Validation binary_cross_entropy = 0.741738
Epoch 183
Validation binary_cross_entropy = 0.625711
Epoch 184
Loss = 2.2870e-01, PNorm = 66.7444, GNorm = 3.7996, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 0.590199
Epoch 185
Validation binary_cross_entropy = 0.537927
Epoch 186
Validation binary_cross_entropy = 0.553995
Epoch 187
Validation binary_cross_entropy = 0.579386
Epoch 188
Validation binary_cross_entropy = 0.607008
Epoch 189
Loss = 1.0560e-01, PNorm = 66.8375, GNorm = 0.8903, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 0.666027
Epoch 190
Validation binary_cross_entropy = 0.688308
Epoch 191
Validation binary_cross_entropy = 0.703030
Epoch 192
Validation binary_cross_entropy = 0.736917
Epoch 193
Validation binary_cross_entropy = 0.739970
Epoch 194
Loss = 1.3045e-01, PNorm = 66.9184, GNorm = 2.0950, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 0.750743
Epoch 195
Validation binary_cross_entropy = 0.793034
Epoch 196
Validation binary_cross_entropy = 0.852751
Epoch 197
Validation binary_cross_entropy = 0.874709
Epoch 198
Validation binary_cross_entropy = 0.913841
Epoch 199
Loss = 9.6365e-02, PNorm = 66.9964, GNorm = 0.9199, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 0.928183
Epoch 200
Validation binary_cross_entropy = 0.964770
Epoch 201
Validation binary_cross_entropy = 0.992088
Epoch 202
Validation binary_cross_entropy = 0.906736
Epoch 203
Validation binary_cross_entropy = 0.840484
Epoch 204
Loss = 1.5256e-01, PNorm = 67.0575, GNorm = 1.4163, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 0.799154
Epoch 205
Validation binary_cross_entropy = 0.770710
Epoch 206
Validation binary_cross_entropy = 0.694112
Epoch 207
Validation binary_cross_entropy = 0.690733
Epoch 208
Validation binary_cross_entropy = 0.684760
Epoch 209
Loss = 1.5869e-01, PNorm = 67.1452, GNorm = 1.2159, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 0.689744
Epoch 210
Validation binary_cross_entropy = 0.743307
Epoch 211
Validation binary_cross_entropy = 0.740692
Epoch 212
Validation binary_cross_entropy = 0.710989
Epoch 213
Validation binary_cross_entropy = 0.729292
Epoch 214
Loss = 9.0190e-02, PNorm = 67.2293, GNorm = 1.6501, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 0.773046
Epoch 215
Validation binary_cross_entropy = 0.850831
Epoch 216
Validation binary_cross_entropy = 0.977448
Epoch 217
Validation binary_cross_entropy = 1.042622
Epoch 218
Validation binary_cross_entropy = 0.915854
Epoch 219
Loss = 1.1593e-01, PNorm = 67.3031, GNorm = 1.5969, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 0.899561
Epoch 220
Validation binary_cross_entropy = 0.933875
Epoch 221
Validation binary_cross_entropy = 1.011824
Epoch 222
Validation binary_cross_entropy = 0.887529
Epoch 223
Validation binary_cross_entropy = 0.798345
Epoch 224
Loss = 2.6421e-01, PNorm = 67.3826, GNorm = 3.1759, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 0.718095
Epoch 225
Validation binary_cross_entropy = 0.626909
Epoch 226
Validation binary_cross_entropy = 0.623843
Epoch 227
Validation binary_cross_entropy = 0.628944
Epoch 228
Validation binary_cross_entropy = 0.638889
Epoch 229
Loss = 1.1235e-01, PNorm = 67.4683, GNorm = 0.9846, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 0.666587
Epoch 230
Validation binary_cross_entropy = 0.720508
Epoch 231
Validation binary_cross_entropy = 0.802497
Epoch 232
Validation binary_cross_entropy = 0.925541
Epoch 233
Validation binary_cross_entropy = 1.054983
Epoch 234
Loss = 1.2048e-01, PNorm = 67.5443, GNorm = 2.8026, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 0.991731
Epoch 235
Validation binary_cross_entropy = 0.983692
Epoch 236
Validation binary_cross_entropy = 1.019579
Epoch 237
Validation binary_cross_entropy = 1.046572
Epoch 238
Validation binary_cross_entropy = 1.068115
Epoch 239
Loss = 1.0555e-01, PNorm = 67.6055, GNorm = 0.5131, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 1.073397
Epoch 240
Validation binary_cross_entropy = 1.095601
Epoch 241
Validation binary_cross_entropy = 1.019189
Epoch 242
Validation binary_cross_entropy = 0.952670
Epoch 243
Validation binary_cross_entropy = 0.917772
Epoch 244
Loss = 1.6130e-01, PNorm = 67.6691, GNorm = 3.0270, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 0.896089
Epoch 245
Validation binary_cross_entropy = 0.965748
Epoch 246
Validation binary_cross_entropy = 0.921935
Epoch 247
Validation binary_cross_entropy = 0.864383
Epoch 248
Validation binary_cross_entropy = 0.861341
Epoch 249
Loss = 1.2510e-01, PNorm = 67.7294, GNorm = 1.6486, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 0.909285
Epoch 250
Validation binary_cross_entropy = 0.975559
Epoch 251
Validation binary_cross_entropy = 1.029051
Epoch 252
Validation binary_cross_entropy = 1.051942
Epoch 253
Validation binary_cross_entropy = 1.028209
Epoch 254
Loss = 4.6535e-02, PNorm = 67.7896, GNorm = 0.8437, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 1.044429
Epoch 255
Validation binary_cross_entropy = 1.108052
Epoch 256
Validation binary_cross_entropy = 1.217584
Epoch 257
Validation binary_cross_entropy = 1.258634
Epoch 258
Validation binary_cross_entropy = 1.113582
Epoch 259
Loss = 1.9162e-01, PNorm = 67.8421, GNorm = 1.9296, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 0.957873
Epoch 260
Validation binary_cross_entropy = 0.875512
Epoch 261
Validation binary_cross_entropy = 0.863050
Epoch 262
Validation binary_cross_entropy = 0.868029
Epoch 263
Validation binary_cross_entropy = 0.853969
Epoch 264
Loss = 9.6258e-02, PNorm = 67.9025, GNorm = 0.8895, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 0.842791
Epoch 265
Validation binary_cross_entropy = 0.853808
Epoch 266
Validation binary_cross_entropy = 0.926803
Epoch 267
Validation binary_cross_entropy = 1.036205
Epoch 268
Validation binary_cross_entropy = 1.101811
Epoch 269
Loss = 4.7476e-02, PNorm = 67.9559, GNorm = 1.1950, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 1.068694
Epoch 270
Validation binary_cross_entropy = 1.070621
Epoch 271
Validation binary_cross_entropy = 1.080504
Epoch 272
Validation binary_cross_entropy = 1.150066
Epoch 273
Validation binary_cross_entropy = 1.288207
Epoch 274
Loss = 9.9065e-02, PNorm = 68.0014, GNorm = 5.5152, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 1.286754
Epoch 275
Validation binary_cross_entropy = 1.195713
Epoch 276
Validation binary_cross_entropy = 1.163258
Epoch 277
Validation binary_cross_entropy = 1.175258
Epoch 278
Validation binary_cross_entropy = 1.191755
Epoch 279
Loss = 8.4779e-02, PNorm = 68.0459, GNorm = 1.1401, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 1.257287
Epoch 280
Validation binary_cross_entropy = 1.257581
Epoch 281
Validation binary_cross_entropy = 1.193638
Epoch 282
Validation binary_cross_entropy = 1.171147
Epoch 283
Validation binary_cross_entropy = 1.180408
Epoch 284
Loss = 1.3435e-01, PNorm = 68.0841, GNorm = 1.9187, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 1.184433
Epoch 285
Validation binary_cross_entropy = 1.217888
Epoch 286
Validation binary_cross_entropy = 1.231984
Epoch 287
Validation binary_cross_entropy = 1.113955
Epoch 288
Validation binary_cross_entropy = 1.041147
Epoch 289
Loss = 5.8976e-02, PNorm = 68.1353, GNorm = 0.8985, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 1.037846
Epoch 290
Validation binary_cross_entropy = 1.024491
Epoch 291
Validation binary_cross_entropy = 1.065444
Epoch 292
Validation binary_cross_entropy = 1.081721
Epoch 293
Validation binary_cross_entropy = 1.018857
Epoch 294
Loss = 4.7834e-02, PNorm = 68.1844, GNorm = 0.4974, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 0.998081
Epoch 295
Validation binary_cross_entropy = 0.990471
Epoch 296
Validation binary_cross_entropy = 1.017386
Epoch 297
Validation binary_cross_entropy = 1.089213
Epoch 298
Validation binary_cross_entropy = 1.123882
Epoch 299
Loss = 4.1473e-02, PNorm = 68.2332, GNorm = 0.9392, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 1.135837
Model 0 best validation binary_cross_entropy = 0.283916 on epoch 107
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.211122
Ensemble test binary_cross_entropy = 0.211122
Fold 6
Splitting data with seed 6
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.4, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=800, bias=False)
        (W_h): Linear(in_features=800, out_features=800, bias=False)
        (W_o): Linear(in_features=933, out_features=800, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.4, inplace=False)
    (1): Linear(in_features=3100, out_features=800, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.4, inplace=False)
    (4): Linear(in_features=800, out_features=800, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.4, inplace=False)
    (7): Linear(in_features=800, out_features=1, bias=True)
  )
)
Number of parameters = 4,627,201
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 0.293489
Epoch 1
Validation binary_cross_entropy = 0.301747
Epoch 2
Validation binary_cross_entropy = 0.614025
Epoch 3
Validation binary_cross_entropy = 0.314358
Epoch 4
Loss = 9.0234e-01, PNorm = 63.1523, GNorm = 8.5275, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 0.292968
Epoch 5
Validation binary_cross_entropy = 0.569553
Epoch 6
Validation binary_cross_entropy = 0.701773
Epoch 7
Validation binary_cross_entropy = 0.312632
Epoch 8
Validation binary_cross_entropy = 0.472939
Epoch 9
Loss = 7.2079e-01, PNorm = 63.2846, GNorm = 5.3228, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 0.644445
Epoch 10
Validation binary_cross_entropy = 0.339892
Epoch 11
Validation binary_cross_entropy = 0.354199
Epoch 12
Validation binary_cross_entropy = 0.683794
Epoch 13
Validation binary_cross_entropy = 0.325539
Epoch 14
Loss = 7.7695e-01, PNorm = 63.4385, GNorm = 5.8793, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 0.317181
Epoch 15
Validation binary_cross_entropy = 1.107939
Epoch 16
Validation binary_cross_entropy = 0.365324
Epoch 17
Validation binary_cross_entropy = 0.296562
Epoch 18
Validation binary_cross_entropy = 0.521165
Epoch 19
Loss = 7.6414e-01, PNorm = 63.5639, GNorm = 6.9624, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 1.012209
Epoch 20
Validation binary_cross_entropy = 0.359591
Epoch 21
Validation binary_cross_entropy = 0.302680
Epoch 22
Validation binary_cross_entropy = 0.396236
Epoch 23
Validation binary_cross_entropy = 0.675818
Epoch 24
Loss = 6.7859e-01, PNorm = 63.6587, GNorm = 4.6212, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 0.603887
Epoch 25
Validation binary_cross_entropy = 0.344314
Epoch 26
Validation binary_cross_entropy = 0.344532
Epoch 27
Validation binary_cross_entropy = 0.449716
Epoch 28
Validation binary_cross_entropy = 0.545673
Epoch 29
Loss = 6.6723e-01, PNorm = 63.7292, GNorm = 2.7616, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 0.454104
Epoch 30
Validation binary_cross_entropy = 0.403414
Epoch 31
Validation binary_cross_entropy = 0.404294
Epoch 32
Validation binary_cross_entropy = 0.496384
Epoch 33
Validation binary_cross_entropy = 0.497117
Epoch 34
Loss = 5.5663e-01, PNorm = 63.7932, GNorm = 2.4544, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 0.546613
Epoch 35
Validation binary_cross_entropy = 0.527903
Epoch 36
Validation binary_cross_entropy = 0.482495
Epoch 37
Validation binary_cross_entropy = 0.515088
Epoch 38
Validation binary_cross_entropy = 0.480556
Epoch 39
Loss = 6.6443e-01, PNorm = 63.8496, GNorm = 3.0463, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 0.417855
Epoch 40
Validation binary_cross_entropy = 0.428011
Epoch 41
Validation binary_cross_entropy = 0.596220
Epoch 42
Validation binary_cross_entropy = 0.606077
Epoch 43
Validation binary_cross_entropy = 0.438176
Epoch 44
Loss = 5.6428e-01, PNorm = 63.9044, GNorm = 3.1183, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 0.406244
Epoch 45
Validation binary_cross_entropy = 0.545882
Epoch 46
Validation binary_cross_entropy = 0.577568
Epoch 47
Validation binary_cross_entropy = 0.458864
Epoch 48
Validation binary_cross_entropy = 0.465031
Epoch 49
Loss = 5.6304e-01, PNorm = 63.9610, GNorm = 2.1331, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 0.493329
Epoch 50
Validation binary_cross_entropy = 0.531007
Epoch 51
Validation binary_cross_entropy = 0.533820
Epoch 52
Validation binary_cross_entropy = 0.491103
Epoch 53
Validation binary_cross_entropy = 0.419268
Epoch 54
Loss = 5.3976e-01, PNorm = 64.0153, GNorm = 2.2686, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 0.440039
Epoch 55
Validation binary_cross_entropy = 0.537297
Epoch 56
Validation binary_cross_entropy = 0.549532
Epoch 57
Validation binary_cross_entropy = 0.469908
Epoch 58
Validation binary_cross_entropy = 0.411494
Epoch 59
Loss = 6.4141e-01, PNorm = 64.0731, GNorm = 2.3230, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 0.447328
Epoch 60
Validation binary_cross_entropy = 0.597291
Epoch 61
Validation binary_cross_entropy = 0.660272
Epoch 62
Validation binary_cross_entropy = 0.548013
Epoch 63
Validation binary_cross_entropy = 0.441398
Epoch 64
Loss = 6.0195e-01, PNorm = 64.1460, GNorm = 2.3538, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 0.392233
Epoch 65
Validation binary_cross_entropy = 0.438474
Epoch 66
Validation binary_cross_entropy = 0.580010
Epoch 67
Validation binary_cross_entropy = 0.697825
Epoch 68
Validation binary_cross_entropy = 0.525521
Epoch 69
Loss = 5.3425e-01, PNorm = 64.2159, GNorm = 2.1359, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 0.393212
Epoch 70
Validation binary_cross_entropy = 0.409103
Epoch 71
Validation binary_cross_entropy = 0.491681
Epoch 72
Validation binary_cross_entropy = 0.414319
Epoch 73
Validation binary_cross_entropy = 0.476861
Epoch 74
Loss = 5.8340e-01, PNorm = 64.2903, GNorm = 2.1058, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 0.682040
Epoch 75
Validation binary_cross_entropy = 0.647853
Epoch 76
Validation binary_cross_entropy = 0.420309
Epoch 77
Validation binary_cross_entropy = 0.343876
Epoch 78
Validation binary_cross_entropy = 0.610494
Epoch 79
Loss = 6.4652e-01, PNorm = 64.3904, GNorm = 3.0685, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 0.288250
Epoch 80
Validation binary_cross_entropy = 0.387944
Epoch 81
Validation binary_cross_entropy = 0.597684
Epoch 82
Validation binary_cross_entropy = 0.712290
Epoch 83
Validation binary_cross_entropy = 0.592159
Epoch 84
Loss = 5.2166e-01, PNorm = 64.4952, GNorm = 1.4487, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 0.446120
Epoch 85
Validation binary_cross_entropy = 0.394136
Epoch 86
Validation binary_cross_entropy = 0.471795
Epoch 87
Validation binary_cross_entropy = 0.539885
Epoch 88
Validation binary_cross_entropy = 0.479185
Epoch 89
Loss = 4.4481e-01, PNorm = 64.6165, GNorm = 1.6449, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 0.408270
Epoch 90
Validation binary_cross_entropy = 0.425316
Epoch 91
Validation binary_cross_entropy = 0.495722
Epoch 92
Validation binary_cross_entropy = 0.386988
Epoch 93
Validation binary_cross_entropy = 0.410314
Epoch 94
Loss = 3.9783e-01, PNorm = 64.7317, GNorm = 2.5933, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 0.363628
Epoch 95
Validation binary_cross_entropy = 0.378598
Epoch 96
Validation binary_cross_entropy = 0.472420
Epoch 97
Validation binary_cross_entropy = 0.365591
Epoch 98
Validation binary_cross_entropy = 0.430086
Epoch 99
Loss = 3.8539e-01, PNorm = 64.8403, GNorm = 1.8466, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.459187
Epoch 100
Validation binary_cross_entropy = 0.399705
Epoch 101
Validation binary_cross_entropy = 0.397056
Epoch 102
Validation binary_cross_entropy = 0.556238
Epoch 103
Validation binary_cross_entropy = 0.377706
Epoch 104
Loss = 3.2994e-01, PNorm = 64.9784, GNorm = 2.0063, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.589278
Epoch 105
Validation binary_cross_entropy = 0.305979
Epoch 106
Validation binary_cross_entropy = 0.376418
Epoch 107
Validation binary_cross_entropy = 0.469289
Epoch 108
Validation binary_cross_entropy = 0.337849
Epoch 109
Loss = 3.4071e-01, PNorm = 65.1220, GNorm = 1.5729, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.324955
Epoch 110
Validation binary_cross_entropy = 0.465295
Epoch 111
Validation binary_cross_entropy = 0.335518
Epoch 112
Validation binary_cross_entropy = 0.303083
Epoch 113
Validation binary_cross_entropy = 0.413269
Epoch 114
Loss = 3.2908e-01, PNorm = 65.2556, GNorm = 1.8398, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.335452
Epoch 115
Validation binary_cross_entropy = 0.301535
Epoch 116
Validation binary_cross_entropy = 0.520122
Epoch 117
Validation binary_cross_entropy = 0.366098
Epoch 118
Validation binary_cross_entropy = 0.312534
Epoch 119
Loss = 3.1951e-01, PNorm = 65.4008, GNorm = 2.6826, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.387874
Epoch 120
Validation binary_cross_entropy = 0.470964
Epoch 121
Validation binary_cross_entropy = 0.385001
Epoch 122
Validation binary_cross_entropy = 0.404180
Epoch 123
Validation binary_cross_entropy = 0.415754
Epoch 124
Loss = 2.8206e-01, PNorm = 65.5385, GNorm = 1.8658, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.418048
Epoch 125
Validation binary_cross_entropy = 0.387753
Epoch 126
Validation binary_cross_entropy = 0.417011
Epoch 127
Validation binary_cross_entropy = 0.365134
Epoch 128
Validation binary_cross_entropy = 0.374518
Epoch 129
Loss = 2.4043e-01, PNorm = 65.6751, GNorm = 1.8044, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.420426
Epoch 130
Validation binary_cross_entropy = 0.451899
Epoch 131
Validation binary_cross_entropy = 0.512231
Epoch 132
Validation binary_cross_entropy = 0.508906
Epoch 133
Validation binary_cross_entropy = 0.396655
Epoch 134
Loss = 2.0624e-01, PNorm = 65.7930, GNorm = 3.7620, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.368230
Epoch 135
Validation binary_cross_entropy = 0.370034
Epoch 136
Validation binary_cross_entropy = 0.375058
Epoch 137
Validation binary_cross_entropy = 0.389770
Epoch 138
Validation binary_cross_entropy = 0.430091
Epoch 139
Loss = 2.7396e-01, PNorm = 65.9172, GNorm = 3.2256, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.472394
Epoch 140
Validation binary_cross_entropy = 0.529886
Epoch 141
Validation binary_cross_entropy = 0.490586
Epoch 142
Validation binary_cross_entropy = 0.475437
Epoch 143
Validation binary_cross_entropy = 0.475941
Epoch 144
Loss = 2.0885e-01, PNorm = 66.0169, GNorm = 1.4926, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.433985
Epoch 145
Validation binary_cross_entropy = 0.439605
Epoch 146
Validation binary_cross_entropy = 0.454332
Epoch 147
Validation binary_cross_entropy = 0.497678
Epoch 148
Validation binary_cross_entropy = 0.543095
Epoch 149
Loss = 1.8164e-01, PNorm = 66.1313, GNorm = 2.3805, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.623861
Epoch 150
Validation binary_cross_entropy = 0.595238
Epoch 151
Validation binary_cross_entropy = 0.583996
Epoch 152
Validation binary_cross_entropy = 0.694042
Epoch 153
Validation binary_cross_entropy = 0.432960
Epoch 154
Loss = 3.0347e-01, PNorm = 66.2375, GNorm = 4.2439, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.425166
Epoch 155
Validation binary_cross_entropy = 0.375888
Epoch 156
Validation binary_cross_entropy = 0.420325
Epoch 157
Validation binary_cross_entropy = 0.466099
Epoch 158
Validation binary_cross_entropy = 0.536407
Epoch 159
Loss = 2.7980e-01, PNorm = 66.3818, GNorm = 1.2861, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.493207
Epoch 160
Validation binary_cross_entropy = 0.453295
Epoch 161
Validation binary_cross_entropy = 0.398143
Epoch 162
Validation binary_cross_entropy = 0.408703
Epoch 163
Validation binary_cross_entropy = 0.420514
Epoch 164
Loss = 1.6335e-01, PNorm = 66.4884, GNorm = 0.9692, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.457490
Epoch 165
Validation binary_cross_entropy = 0.526412
Epoch 166
Validation binary_cross_entropy = 0.555573
Epoch 167
Validation binary_cross_entropy = 0.598804
Epoch 168
Validation binary_cross_entropy = 0.684543
Epoch 169
Loss = 1.2683e-01, PNorm = 66.5932, GNorm = 2.1735, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.732442
Epoch 170
Validation binary_cross_entropy = 0.769169
Epoch 171
Validation binary_cross_entropy = 0.779376
Epoch 172
Validation binary_cross_entropy = 0.823284
Epoch 173
Validation binary_cross_entropy = 0.650657
Epoch 174
Loss = 1.7297e-01, PNorm = 66.6783, GNorm = 2.7252, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 0.623276
Epoch 175
Validation binary_cross_entropy = 0.513569
Epoch 176
Validation binary_cross_entropy = 0.495311
Epoch 177
Validation binary_cross_entropy = 0.538038
Epoch 178
Validation binary_cross_entropy = 0.600122
Epoch 179
Loss = 2.0669e-01, PNorm = 66.7669, GNorm = 2.9143, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 0.588863
Epoch 180
Validation binary_cross_entropy = 0.547671
Epoch 181
Validation binary_cross_entropy = 0.551308
Epoch 182
Validation binary_cross_entropy = 0.580595
Epoch 183
Validation binary_cross_entropy = 0.653159
Epoch 184
Loss = 1.1277e-01, PNorm = 66.8584, GNorm = 1.9201, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 0.670056
Epoch 185
Validation binary_cross_entropy = 0.690236
Epoch 186
Validation binary_cross_entropy = 0.730153
Epoch 187
Validation binary_cross_entropy = 0.697845
Epoch 188
Validation binary_cross_entropy = 0.709235
Epoch 189
Loss = 1.4351e-01, PNorm = 66.9443, GNorm = 1.6607, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 0.682350
Epoch 190
Validation binary_cross_entropy = 0.683389
Epoch 191
Validation binary_cross_entropy = 0.713247
Epoch 192
Validation binary_cross_entropy = 0.718493
Epoch 193
Validation binary_cross_entropy = 0.732184
Epoch 194
Loss = 9.9132e-02, PNorm = 67.0187, GNorm = 1.3884, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 0.761181
Epoch 195
Validation binary_cross_entropy = 0.778129
Epoch 196
Validation binary_cross_entropy = 0.787114
Epoch 197
Validation binary_cross_entropy = 0.803544
Epoch 198
Validation binary_cross_entropy = 0.781996
Epoch 199
Loss = 7.4685e-02, PNorm = 67.0802, GNorm = 0.5584, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 0.653189
Epoch 200
Validation binary_cross_entropy = 0.634238
Epoch 201
Validation binary_cross_entropy = 0.564254
Epoch 202
Validation binary_cross_entropy = 0.562341
Epoch 203
Validation binary_cross_entropy = 0.580927
Epoch 204
Loss = 1.9451e-01, PNorm = 67.1443, GNorm = 0.5528, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 0.579791
Epoch 205
Validation binary_cross_entropy = 0.606667
Epoch 206
Validation binary_cross_entropy = 0.614933
Epoch 207
Validation binary_cross_entropy = 0.645336
Epoch 208
Validation binary_cross_entropy = 0.714849
Epoch 209
Loss = 9.1322e-02, PNorm = 67.2126, GNorm = 1.1719, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 0.756040
Epoch 210
Validation binary_cross_entropy = 0.703131
Epoch 211
Validation binary_cross_entropy = 0.702410
Epoch 212
Validation binary_cross_entropy = 0.740334
Epoch 213
Validation binary_cross_entropy = 0.795591
Epoch 214
Loss = 1.0825e-01, PNorm = 67.2867, GNorm = 1.2293, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 0.776975
Epoch 215
Validation binary_cross_entropy = 0.746601
Epoch 216
Validation binary_cross_entropy = 0.747185
Epoch 217
Validation binary_cross_entropy = 0.761786
Epoch 218
Validation binary_cross_entropy = 0.779205
Epoch 219
Loss = 4.9489e-02, PNorm = 67.3589, GNorm = 1.6405, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 0.769287
Epoch 220
Validation binary_cross_entropy = 0.754499
Epoch 221
Validation binary_cross_entropy = 0.772852
Epoch 222
Validation binary_cross_entropy = 0.803626
Epoch 223
Validation binary_cross_entropy = 0.815199
Epoch 224
Loss = 6.0981e-02, PNorm = 67.4177, GNorm = 1.4392, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 0.793646
Epoch 225
Validation binary_cross_entropy = 0.771697
Epoch 226
Validation binary_cross_entropy = 0.769642
Epoch 227
Validation binary_cross_entropy = 0.821159
Epoch 228
Validation binary_cross_entropy = 0.826643
Epoch 229
Loss = 2.1846e-02, PNorm = 67.4676, GNorm = 0.4477, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 0.796813
Epoch 230
Validation binary_cross_entropy = 0.786348
Epoch 231
Validation binary_cross_entropy = 0.783144
Epoch 232
Validation binary_cross_entropy = 0.780293
Epoch 233
Validation binary_cross_entropy = 0.777062
Epoch 234
Loss = 1.0163e-01, PNorm = 67.5226, GNorm = 1.9619, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 0.831205
Epoch 235
Validation binary_cross_entropy = 0.811773
Epoch 236
Validation binary_cross_entropy = 0.748357
Epoch 237
Validation binary_cross_entropy = 0.740677
Epoch 238
Validation binary_cross_entropy = 0.759986
Epoch 239
Loss = 4.7129e-02, PNorm = 67.5819, GNorm = 0.6335, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 0.786416
Epoch 240
Validation binary_cross_entropy = 0.821375
Epoch 241
Validation binary_cross_entropy = 0.879624
Epoch 242
Validation binary_cross_entropy = 0.891403
Epoch 243
Validation binary_cross_entropy = 0.874506
Epoch 244
Loss = 4.4674e-02, PNorm = 67.6467, GNorm = 0.3946, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 0.816510
Epoch 245
Validation binary_cross_entropy = 0.785812
Epoch 246
Validation binary_cross_entropy = 0.822601
Epoch 247
Validation binary_cross_entropy = 0.839148
Epoch 248
Validation binary_cross_entropy = 0.797250
Epoch 249
Loss = 3.1088e-02, PNorm = 67.7285, GNorm = 0.6917, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 0.812645
Epoch 250
Validation binary_cross_entropy = 0.836272
Epoch 251
Validation binary_cross_entropy = 0.905774
Epoch 252
Validation binary_cross_entropy = 0.931950
Epoch 253
Validation binary_cross_entropy = 0.793924
Epoch 254
Loss = 1.0904e-01, PNorm = 67.7938, GNorm = 1.6225, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 0.697587
Epoch 255
Validation binary_cross_entropy = 0.680482
Epoch 256
Validation binary_cross_entropy = 0.684938
Epoch 257
Validation binary_cross_entropy = 0.705648
Epoch 258
Validation binary_cross_entropy = 0.705974
Epoch 259
Loss = 5.6002e-02, PNorm = 67.8455, GNorm = 0.9963, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 0.720226
Epoch 260
Validation binary_cross_entropy = 0.748949
Epoch 261
Validation binary_cross_entropy = 0.790919
Epoch 262
Validation binary_cross_entropy = 0.835619
Epoch 263
Validation binary_cross_entropy = 0.862126
Epoch 264
Loss = 4.5151e-02, PNorm = 67.8979, GNorm = 1.4306, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 0.889121
Epoch 265
Validation binary_cross_entropy = 0.909027
Epoch 266
Validation binary_cross_entropy = 0.931612
Epoch 267
Validation binary_cross_entropy = 0.956948
Epoch 268
Validation binary_cross_entropy = 0.984944
Epoch 269
Loss = 4.6895e-02, PNorm = 67.9426, GNorm = 1.6135, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 1.013820
Epoch 270
Validation binary_cross_entropy = 0.963109
Epoch 271
Validation binary_cross_entropy = 0.990398
Epoch 272
Validation binary_cross_entropy = 0.955213
Epoch 273
Validation binary_cross_entropy = 1.027863
Epoch 274
Loss = 9.8111e-02, PNorm = 67.9907, GNorm = 1.4186, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 1.092378
Epoch 275
Validation binary_cross_entropy = 1.052894
Epoch 276
Validation binary_cross_entropy = 0.980557
Epoch 277
Validation binary_cross_entropy = 0.906990
Epoch 278
Validation binary_cross_entropy = 0.875074
Epoch 279
Loss = 1.0945e-01, PNorm = 68.0396, GNorm = 0.1009, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 0.956885
Epoch 280
Validation binary_cross_entropy = 1.068989
Epoch 281
Validation binary_cross_entropy = 0.979120
Epoch 282
Validation binary_cross_entropy = 0.798740
Epoch 283
Validation binary_cross_entropy = 0.739207
Epoch 284
Loss = 8.1055e-02, PNorm = 68.0874, GNorm = 0.9133, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 0.737170
Epoch 285
Validation binary_cross_entropy = 0.751274
Epoch 286
Validation binary_cross_entropy = 0.768261
Epoch 287
Validation binary_cross_entropy = 0.768341
Epoch 288
Validation binary_cross_entropy = 0.765707
Epoch 289
Loss = 3.9521e-02, PNorm = 68.1373, GNorm = 0.3561, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 0.774728
Epoch 290
Validation binary_cross_entropy = 0.796114
Epoch 291
Validation binary_cross_entropy = 0.830233
Epoch 292
Validation binary_cross_entropy = 0.885370
Epoch 293
Validation binary_cross_entropy = 0.929380
Epoch 294
Loss = 4.8610e-02, PNorm = 68.1756, GNorm = 0.9885, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 0.938708
Epoch 295
Validation binary_cross_entropy = 0.916867
Epoch 296
Validation binary_cross_entropy = 0.910384
Epoch 297
Validation binary_cross_entropy = 0.928141
Epoch 298
Validation binary_cross_entropy = 0.966286
Epoch 299
Loss = 4.0095e-02, PNorm = 68.2091, GNorm = 1.5233, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 1.059284
Model 0 best validation binary_cross_entropy = 0.288250 on epoch 79
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.227363
Ensemble test binary_cross_entropy = 0.227363
Fold 7
Splitting data with seed 7
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.4, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=800, bias=False)
        (W_h): Linear(in_features=800, out_features=800, bias=False)
        (W_o): Linear(in_features=933, out_features=800, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.4, inplace=False)
    (1): Linear(in_features=3100, out_features=800, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.4, inplace=False)
    (4): Linear(in_features=800, out_features=800, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.4, inplace=False)
    (7): Linear(in_features=800, out_features=1, bias=True)
  )
)
Number of parameters = 4,627,201
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 0.287578
Epoch 1
Validation binary_cross_entropy = 0.297397
Epoch 2
Validation binary_cross_entropy = 0.590341
Epoch 3
Validation binary_cross_entropy = 0.310910
Epoch 4
Loss = 9.0203e-01, PNorm = 63.1507, GNorm = 5.0038, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 0.299353
Epoch 5
Validation binary_cross_entropy = 0.396616
Epoch 6
Validation binary_cross_entropy = 0.793094
Epoch 7
Validation binary_cross_entropy = 0.293812
Epoch 8
Validation binary_cross_entropy = 0.344589
Epoch 9
Loss = 7.5659e-01, PNorm = 63.2866, GNorm = 5.1372, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 0.988440
Epoch 10
Validation binary_cross_entropy = 0.341280
Epoch 11
Validation binary_cross_entropy = 0.296808
Epoch 12
Validation binary_cross_entropy = 0.587465
Epoch 13
Validation binary_cross_entropy = 0.562724
Epoch 14
Loss = 7.0592e-01, PNorm = 63.4369, GNorm = 4.6468, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 0.358044
Epoch 15
Validation binary_cross_entropy = 0.408181
Epoch 16
Validation binary_cross_entropy = 0.459692
Epoch 17
Validation binary_cross_entropy = 0.547062
Epoch 18
Validation binary_cross_entropy = 0.555385
Epoch 19
Loss = 6.5802e-01, PNorm = 63.5515, GNorm = 3.5610, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 0.431401
Epoch 20
Validation binary_cross_entropy = 0.434367
Epoch 21
Validation binary_cross_entropy = 0.566763
Epoch 22
Validation binary_cross_entropy = 0.576055
Epoch 23
Validation binary_cross_entropy = 0.375740
Epoch 24
Loss = 6.7032e-01, PNorm = 63.6515, GNorm = 3.1057, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 0.497830
Epoch 25
Validation binary_cross_entropy = 0.461870
Epoch 26
Validation binary_cross_entropy = 0.521824
Epoch 27
Validation binary_cross_entropy = 0.481439
Epoch 28
Validation binary_cross_entropy = 0.457925
Epoch 29
Loss = 6.1084e-01, PNorm = 63.7153, GNorm = 2.8922, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 0.472704
Epoch 30
Validation binary_cross_entropy = 0.517028
Epoch 31
Validation binary_cross_entropy = 0.486613
Epoch 32
Validation binary_cross_entropy = 0.485379
Epoch 33
Validation binary_cross_entropy = 0.468135
Epoch 34
Loss = 5.9811e-01, PNorm = 63.7809, GNorm = 2.4938, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 0.427763
Epoch 35
Validation binary_cross_entropy = 0.469912
Epoch 36
Validation binary_cross_entropy = 0.582947
Epoch 37
Validation binary_cross_entropy = 0.426217
Epoch 38
Validation binary_cross_entropy = 0.360635
Epoch 39
Loss = 6.4190e-01, PNorm = 63.8463, GNorm = 3.5083, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 0.423756
Epoch 40
Validation binary_cross_entropy = 0.641698
Epoch 41
Validation binary_cross_entropy = 0.456837
Epoch 42
Validation binary_cross_entropy = 0.351814
Epoch 43
Validation binary_cross_entropy = 0.423985
Epoch 44
Loss = 5.9507e-01, PNorm = 63.9060, GNorm = 2.3729, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 0.582152
Epoch 45
Validation binary_cross_entropy = 0.512781
Epoch 46
Validation binary_cross_entropy = 0.427940
Epoch 47
Validation binary_cross_entropy = 0.418260
Epoch 48
Validation binary_cross_entropy = 0.491310
Epoch 49
Loss = 6.2303e-01, PNorm = 63.9639, GNorm = 2.3490, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 0.479724
Epoch 50
Validation binary_cross_entropy = 0.441937
Epoch 51
Validation binary_cross_entropy = 0.442449
Epoch 52
Validation binary_cross_entropy = 0.424162
Epoch 53
Validation binary_cross_entropy = 0.467916
Epoch 54
Loss = 5.8621e-01, PNorm = 64.0248, GNorm = 1.7079, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 0.534935
Epoch 55
Validation binary_cross_entropy = 0.497515
Epoch 56
Validation binary_cross_entropy = 0.458755
Epoch 57
Validation binary_cross_entropy = 0.389588
Epoch 58
Validation binary_cross_entropy = 0.409997
Epoch 59
Loss = 5.8892e-01, PNorm = 64.0988, GNorm = 1.9613, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 0.407530
Epoch 60
Validation binary_cross_entropy = 0.524273
Epoch 61
Validation binary_cross_entropy = 0.432197
Epoch 62
Validation binary_cross_entropy = 0.456950
Epoch 63
Validation binary_cross_entropy = 0.508430
Epoch 64
Loss = 5.3164e-01, PNorm = 64.1781, GNorm = 1.6254, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 0.472443
Epoch 65
Validation binary_cross_entropy = 0.359730
Epoch 66
Validation binary_cross_entropy = 0.345241
Epoch 67
Validation binary_cross_entropy = 0.515456
Epoch 68
Validation binary_cross_entropy = 0.433147
Epoch 69
Loss = 4.9257e-01, PNorm = 64.2648, GNorm = 1.4643, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 0.365673
Epoch 70
Validation binary_cross_entropy = 0.412234
Epoch 71
Validation binary_cross_entropy = 0.438998
Epoch 72
Validation binary_cross_entropy = 0.335975
Epoch 73
Validation binary_cross_entropy = 0.418850
Epoch 74
Loss = 4.5810e-01, PNorm = 64.3710, GNorm = 1.7875, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 0.437598
Epoch 75
Validation binary_cross_entropy = 0.341689
Epoch 76
Validation binary_cross_entropy = 0.415183
Epoch 77
Validation binary_cross_entropy = 0.423582
Epoch 78
Validation binary_cross_entropy = 0.330176
Epoch 79
Loss = 4.6943e-01, PNorm = 64.4765, GNorm = 1.8497, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 0.450325
Epoch 80
Validation binary_cross_entropy = 0.415297
Epoch 81
Validation binary_cross_entropy = 0.322829
Epoch 82
Validation binary_cross_entropy = 0.406414
Epoch 83
Validation binary_cross_entropy = 0.519345
Epoch 84
Loss = 5.1618e-01, PNorm = 64.5854, GNorm = 1.7793, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 0.313922
Epoch 85
Validation binary_cross_entropy = 0.443533
Epoch 86
Validation binary_cross_entropy = 0.518932
Epoch 87
Validation binary_cross_entropy = 0.403173
Epoch 88
Validation binary_cross_entropy = 0.378945
Epoch 89
Loss = 4.0232e-01, PNorm = 64.7009, GNorm = 1.8410, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 0.460353
Epoch 90
Validation binary_cross_entropy = 0.357802
Epoch 91
Validation binary_cross_entropy = 0.298252
Epoch 92
Validation binary_cross_entropy = 0.688198
Epoch 93
Validation binary_cross_entropy = 0.446243
Epoch 94
Loss = 3.8489e-01, PNorm = 64.8121, GNorm = 1.4452, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 0.321528
Epoch 95
Validation binary_cross_entropy = 0.336616
Epoch 96
Validation binary_cross_entropy = 0.614482
Epoch 97
Validation binary_cross_entropy = 0.421732
Epoch 98
Validation binary_cross_entropy = 0.309133
Epoch 99
Loss = 3.8713e-01, PNorm = 64.9433, GNorm = 1.7246, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.412696
Epoch 100
Validation binary_cross_entropy = 0.487821
Epoch 101
Validation binary_cross_entropy = 0.368465
Epoch 102
Validation binary_cross_entropy = 0.354444
Epoch 103
Validation binary_cross_entropy = 0.438060
Epoch 104
Loss = 3.3578e-01, PNorm = 65.0739, GNorm = 1.5572, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.409859
Epoch 105
Validation binary_cross_entropy = 0.347598
Epoch 106
Validation binary_cross_entropy = 0.414044
Epoch 107
Validation binary_cross_entropy = 0.397244
Epoch 108
Validation binary_cross_entropy = 0.334173
Epoch 109
Loss = 3.3298e-01, PNorm = 65.1985, GNorm = 2.6815, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.513396
Epoch 110
Validation binary_cross_entropy = 0.483032
Epoch 111
Validation binary_cross_entropy = 0.370771
Epoch 112
Validation binary_cross_entropy = 0.475437
Epoch 113
Validation binary_cross_entropy = 0.522656
Epoch 114
Loss = 3.4429e-01, PNorm = 65.3315, GNorm = 1.3007, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.379143
Epoch 115
Validation binary_cross_entropy = 0.430793
Epoch 116
Validation binary_cross_entropy = 0.527545
Epoch 117
Validation binary_cross_entropy = 0.359518
Epoch 118
Validation binary_cross_entropy = 0.321336
Epoch 119
Loss = 4.1713e-01, PNorm = 65.4728, GNorm = 1.4663, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.378599
Epoch 120
Validation binary_cross_entropy = 0.492115
Epoch 121
Validation binary_cross_entropy = 0.388650
Epoch 122
Validation binary_cross_entropy = 0.424830
Epoch 123
Validation binary_cross_entropy = 0.495568
Epoch 124
Loss = 2.6186e-01, PNorm = 65.6155, GNorm = 1.0727, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.512523
Epoch 125
Validation binary_cross_entropy = 0.550059
Epoch 126
Validation binary_cross_entropy = 0.543183
Epoch 127
Validation binary_cross_entropy = 0.542042
Epoch 128
Validation binary_cross_entropy = 0.507629
Epoch 129
Loss = 1.7519e-01, PNorm = 65.7447, GNorm = 1.4237, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.488613
Epoch 130
Validation binary_cross_entropy = 0.484101
Epoch 131
Validation binary_cross_entropy = 0.480331
Epoch 132
Validation binary_cross_entropy = 0.526641
Epoch 133
Validation binary_cross_entropy = 0.642370
Epoch 134
Loss = 2.8842e-01, PNorm = 65.8746, GNorm = 3.5129, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.527713
Epoch 135
Validation binary_cross_entropy = 0.502124
Epoch 136
Validation binary_cross_entropy = 0.554298
Epoch 137
Validation binary_cross_entropy = 0.489662
Epoch 138
Validation binary_cross_entropy = 0.496444
Epoch 139
Loss = 2.2473e-01, PNorm = 66.0105, GNorm = 1.1665, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.578672
Epoch 140
Validation binary_cross_entropy = 0.674068
Epoch 141
Validation binary_cross_entropy = 0.579720
Epoch 142
Validation binary_cross_entropy = 0.600954
Epoch 143
Validation binary_cross_entropy = 0.671611
Epoch 144
Loss = 1.8399e-01, PNorm = 66.1370, GNorm = 1.9431, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.705063
Epoch 145
Validation binary_cross_entropy = 0.745262
Epoch 146
Validation binary_cross_entropy = 0.791273
Epoch 147
Validation binary_cross_entropy = 0.786166
Epoch 148
Validation binary_cross_entropy = 0.715535
Epoch 149
Loss = 1.6662e-01, PNorm = 66.2380, GNorm = 3.7739, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.675608
Epoch 150
Validation binary_cross_entropy = 0.587579
Epoch 151
Validation binary_cross_entropy = 0.535914
Epoch 152
Validation binary_cross_entropy = 0.518489
Epoch 153
Validation binary_cross_entropy = 0.515610
Epoch 154
Loss = 1.7910e-01, PNorm = 66.3425, GNorm = 1.7638, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.554122
Epoch 155
Validation binary_cross_entropy = 0.570995
Epoch 156
Validation binary_cross_entropy = 0.605536
Epoch 157
Validation binary_cross_entropy = 0.608824
Epoch 158
Validation binary_cross_entropy = 0.664970
Epoch 159
Loss = 1.8547e-01, PNorm = 66.4384, GNorm = 3.0301, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.659517
Epoch 160
Validation binary_cross_entropy = 0.678668
Epoch 161
Validation binary_cross_entropy = 0.698399
Epoch 162
Validation binary_cross_entropy = 0.696986
Epoch 163
Validation binary_cross_entropy = 0.682270
Epoch 164
Loss = 1.3554e-01, PNorm = 66.5338, GNorm = 3.5417, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.699160
Epoch 165
Validation binary_cross_entropy = 0.759140
Epoch 166
Validation binary_cross_entropy = 0.697140
Epoch 167
Validation binary_cross_entropy = 0.667196
Epoch 168
Validation binary_cross_entropy = 0.706802
Epoch 169
Loss = 1.8584e-01, PNorm = 66.6263, GNorm = 0.6898, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.774440
Epoch 170
Validation binary_cross_entropy = 0.774215
Epoch 171
Validation binary_cross_entropy = 0.745034
Epoch 172
Validation binary_cross_entropy = 0.737186
Epoch 173
Validation binary_cross_entropy = 0.786655
Epoch 174
Loss = 1.4078e-01, PNorm = 66.7301, GNorm = 1.3500, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 0.806647
Epoch 175
Validation binary_cross_entropy = 0.800242
Epoch 176
Validation binary_cross_entropy = 0.921135
Epoch 177
Validation binary_cross_entropy = 0.701197
Epoch 178
Validation binary_cross_entropy = 0.594028
Epoch 179
Loss = 1.3529e-01, PNorm = 66.8368, GNorm = 1.0233, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 0.568053
Epoch 180
Validation binary_cross_entropy = 0.536981
Epoch 181
Validation binary_cross_entropy = 0.557621
Epoch 182
Validation binary_cross_entropy = 0.596732
Epoch 183
Validation binary_cross_entropy = 0.630400
Epoch 184
Loss = 9.7319e-02, PNorm = 66.9607, GNorm = 1.3460, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 0.683154
Epoch 185
Validation binary_cross_entropy = 0.720070
Epoch 186
Validation binary_cross_entropy = 0.752898
Epoch 187
Validation binary_cross_entropy = 0.803656
Epoch 188
Validation binary_cross_entropy = 0.788732
Epoch 189
Loss = 6.4205e-02, PNorm = 67.0530, GNorm = 1.7694, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 0.864521
Epoch 190
Validation binary_cross_entropy = 0.801609
Epoch 191
Validation binary_cross_entropy = 0.835918
Epoch 192
Validation binary_cross_entropy = 0.799484
Epoch 193
Validation binary_cross_entropy = 0.710151
Epoch 194
Loss = 1.2747e-01, PNorm = 67.1372, GNorm = 2.8043, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 0.713651
Epoch 195
Validation binary_cross_entropy = 0.671399
Epoch 196
Validation binary_cross_entropy = 0.731035
Epoch 197
Validation binary_cross_entropy = 0.731456
Epoch 198
Validation binary_cross_entropy = 0.703867
Epoch 199
Loss = 2.2353e-01, PNorm = 67.2191, GNorm = 4.1797, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 0.735032
Epoch 200
Validation binary_cross_entropy = 0.875234
Epoch 201
Validation binary_cross_entropy = 0.845570
Epoch 202
Validation binary_cross_entropy = 0.729508
Epoch 203
Validation binary_cross_entropy = 0.694990
Epoch 204
Loss = 1.1624e-01, PNorm = 67.3057, GNorm = 1.7970, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 0.709775
Epoch 205
Validation binary_cross_entropy = 0.673603
Epoch 206
Validation binary_cross_entropy = 0.686100
Epoch 207
Validation binary_cross_entropy = 0.640591
Epoch 208
Validation binary_cross_entropy = 0.622091
Epoch 209
Loss = 8.8846e-02, PNorm = 67.3843, GNorm = 1.1637, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 0.629825
Epoch 210
Validation binary_cross_entropy = 0.669578
Epoch 211
Validation binary_cross_entropy = 0.714685
Epoch 212
Validation binary_cross_entropy = 0.712062
Epoch 213
Validation binary_cross_entropy = 0.733969
Epoch 214
Loss = 1.2771e-01, PNorm = 67.4633, GNorm = 0.7641, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 0.775567
Epoch 215
Validation binary_cross_entropy = 0.808647
Epoch 216
Validation binary_cross_entropy = 0.854152
Epoch 217
Validation binary_cross_entropy = 0.845440
Epoch 218
Validation binary_cross_entropy = 0.766352
Epoch 219
Loss = 6.2457e-02, PNorm = 67.5301, GNorm = 1.2830, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 0.751419
Epoch 220
Validation binary_cross_entropy = 0.756421
Epoch 221
Validation binary_cross_entropy = 0.770285
Epoch 222
Validation binary_cross_entropy = 0.796439
Epoch 223
Validation binary_cross_entropy = 0.810559
Epoch 224
Loss = 6.6946e-02, PNorm = 67.5919, GNorm = 0.7677, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 0.821371
Epoch 225
Validation binary_cross_entropy = 0.856219
Epoch 226
Validation binary_cross_entropy = 0.921635
Epoch 227
Validation binary_cross_entropy = 0.988436
Epoch 228
Validation binary_cross_entropy = 1.015962
Epoch 229
Loss = 5.4501e-02, PNorm = 67.6420, GNorm = 1.6215, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 1.064670
Epoch 230
Validation binary_cross_entropy = 1.100842
Epoch 231
Validation binary_cross_entropy = 1.110163
Epoch 232
Validation binary_cross_entropy = 1.103050
Epoch 233
Validation binary_cross_entropy = 1.117529
Epoch 234
Loss = 7.8469e-02, PNorm = 67.6957, GNorm = 2.0957, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 0.979556
Epoch 235
Validation binary_cross_entropy = 0.891325
Epoch 236
Validation binary_cross_entropy = 0.824478
Epoch 237
Validation binary_cross_entropy = 0.802508
Epoch 238
Validation binary_cross_entropy = 0.815035
Epoch 239
Loss = 1.4276e-01, PNorm = 67.7460, GNorm = 1.5064, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 0.753825
Epoch 240
Validation binary_cross_entropy = 0.703224
Epoch 241
Validation binary_cross_entropy = 0.715579
Epoch 242
Validation binary_cross_entropy = 0.769384
Epoch 243
Validation binary_cross_entropy = 0.816311
Epoch 244
Loss = 9.6183e-02, PNorm = 67.8153, GNorm = 0.8879, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 0.879800
Epoch 245
Validation binary_cross_entropy = 0.951524
Epoch 246
Validation binary_cross_entropy = 0.981683
Epoch 247
Validation binary_cross_entropy = 1.051914
Epoch 248
Validation binary_cross_entropy = 1.095258
Epoch 249
Loss = 4.7169e-02, PNorm = 67.8732, GNorm = 0.4464, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 1.024466
Epoch 250
Validation binary_cross_entropy = 1.004154
Epoch 251
Validation binary_cross_entropy = 0.967364
Epoch 252
Validation binary_cross_entropy = 0.908230
Epoch 253
Validation binary_cross_entropy = 0.852860
Epoch 254
Loss = 6.3485e-02, PNorm = 67.9236, GNorm = 1.0660, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 0.820879
Epoch 255
Validation binary_cross_entropy = 0.795674
Epoch 256
Validation binary_cross_entropy = 0.789801
Epoch 257
Validation binary_cross_entropy = 0.793679
Epoch 258
Validation binary_cross_entropy = 0.820466
Epoch 259
Loss = 9.9206e-02, PNorm = 67.9862, GNorm = 2.0106, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 0.887989
Epoch 260
Validation binary_cross_entropy = 0.916336
Epoch 261
Validation binary_cross_entropy = 0.916691
Epoch 262
Validation binary_cross_entropy = 0.925399
Epoch 263
Validation binary_cross_entropy = 0.965837
Epoch 264
Loss = 1.1548e-01, PNorm = 68.0462, GNorm = 3.4384, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 0.973522
Epoch 265
Validation binary_cross_entropy = 1.023698
Epoch 266
Validation binary_cross_entropy = 1.039478
Epoch 267
Validation binary_cross_entropy = 1.005325
Epoch 268
Validation binary_cross_entropy = 1.015441
Epoch 269
Loss = 6.5457e-02, PNorm = 68.0935, GNorm = 1.0933, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 1.009015
Epoch 270
Validation binary_cross_entropy = 1.012466
Epoch 271
Validation binary_cross_entropy = 1.006424
Epoch 272
Validation binary_cross_entropy = 0.961708
Epoch 273
Validation binary_cross_entropy = 0.943437
Epoch 274
Loss = 6.7687e-02, PNorm = 68.1480, GNorm = 0.8594, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 0.930276
Epoch 275
Validation binary_cross_entropy = 0.929971
Epoch 276
Validation binary_cross_entropy = 0.924498
Epoch 277
Validation binary_cross_entropy = 0.926440
Epoch 278
Validation binary_cross_entropy = 0.926770
Epoch 279
Loss = 6.4921e-02, PNorm = 68.2038, GNorm = 1.6578, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 1.014007
Epoch 280
Validation binary_cross_entropy = 0.942076
Epoch 281
Validation binary_cross_entropy = 1.022825
Epoch 282
Validation binary_cross_entropy = 0.938791
Epoch 283
Validation binary_cross_entropy = 0.919641
Epoch 284
Loss = 8.2633e-02, PNorm = 68.2679, GNorm = 2.9245, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 0.880076
Epoch 285
Validation binary_cross_entropy = 0.867580
Epoch 286
Validation binary_cross_entropy = 0.888576
Epoch 287
Validation binary_cross_entropy = 0.794869
Epoch 288
Validation binary_cross_entropy = 0.758853
Epoch 289
Loss = 8.0506e-02, PNorm = 68.3306, GNorm = 0.9081, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 0.747756
Epoch 290
Validation binary_cross_entropy = 0.727784
Epoch 291
Validation binary_cross_entropy = 0.755987
Epoch 292
Validation binary_cross_entropy = 0.776505
Epoch 293
Validation binary_cross_entropy = 0.787216
Epoch 294
Loss = 2.4763e-02, PNorm = 68.3990, GNorm = 0.8119, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 0.834675
Epoch 295
Validation binary_cross_entropy = 0.865502
Epoch 296
Validation binary_cross_entropy = 0.914333
Epoch 297
Validation binary_cross_entropy = 0.958716
Epoch 298
Validation binary_cross_entropy = 0.988244
Epoch 299
Loss = 5.5585e-02, PNorm = 68.4531, GNorm = 0.7104, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 0.999965
Model 0 best validation binary_cross_entropy = 0.287578 on epoch 0
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.223729
Ensemble test binary_cross_entropy = 0.223729
Fold 8
Splitting data with seed 8
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.4, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=800, bias=False)
        (W_h): Linear(in_features=800, out_features=800, bias=False)
        (W_o): Linear(in_features=933, out_features=800, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.4, inplace=False)
    (1): Linear(in_features=3100, out_features=800, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.4, inplace=False)
    (4): Linear(in_features=800, out_features=800, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.4, inplace=False)
    (7): Linear(in_features=800, out_features=1, bias=True)
  )
)
Number of parameters = 4,627,201
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 0.287546
Epoch 1
Validation binary_cross_entropy = 0.301045
Epoch 2
Validation binary_cross_entropy = 0.583419
Epoch 3
Validation binary_cross_entropy = 0.319260
Epoch 4
Loss = 8.4577e-01, PNorm = 63.1524, GNorm = 7.6351, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 0.297064
Epoch 5
Validation binary_cross_entropy = 0.525440
Epoch 6
Validation binary_cross_entropy = 0.813832
Epoch 7
Validation binary_cross_entropy = 0.322716
Epoch 8
Validation binary_cross_entropy = 0.430601
Epoch 9
Loss = 6.5036e-01, PNorm = 63.2942, GNorm = 4.3828, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 0.589930
Epoch 10
Validation binary_cross_entropy = 0.347977
Epoch 11
Validation binary_cross_entropy = 0.523647
Epoch 12
Validation binary_cross_entropy = 0.731850
Epoch 13
Validation binary_cross_entropy = 0.302254
Epoch 14
Loss = 9.3429e-01, PNorm = 63.4532, GNorm = 6.9298, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 0.343261
Epoch 15
Validation binary_cross_entropy = 0.570982
Epoch 16
Validation binary_cross_entropy = 0.376336
Epoch 17
Validation binary_cross_entropy = 0.437099
Epoch 18
Validation binary_cross_entropy = 0.617986
Epoch 19
Loss = 7.1439e-01, PNorm = 63.5767, GNorm = 3.5800, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 0.366876
Epoch 20
Validation binary_cross_entropy = 0.333799
Epoch 21
Validation binary_cross_entropy = 0.654548
Epoch 22
Validation binary_cross_entropy = 0.631589
Epoch 23
Validation binary_cross_entropy = 0.338761
Epoch 24
Loss = 7.0166e-01, PNorm = 63.6766, GNorm = 4.7686, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 0.346973
Epoch 25
Validation binary_cross_entropy = 0.618859
Epoch 26
Validation binary_cross_entropy = 0.695850
Epoch 27
Validation binary_cross_entropy = 0.450053
Epoch 28
Validation binary_cross_entropy = 0.402421
Epoch 29
Loss = 5.9487e-01, PNorm = 63.7513, GNorm = 2.6708, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 0.571627
Epoch 30
Validation binary_cross_entropy = 0.708425
Epoch 31
Validation binary_cross_entropy = 0.513279
Epoch 32
Validation binary_cross_entropy = 0.401880
Epoch 33
Validation binary_cross_entropy = 0.388981
Epoch 34
Loss = 6.0567e-01, PNorm = 63.8215, GNorm = 3.1519, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 0.544627
Epoch 35
Validation binary_cross_entropy = 0.651220
Epoch 36
Validation binary_cross_entropy = 0.560333
Epoch 37
Validation binary_cross_entropy = 0.471793
Epoch 38
Validation binary_cross_entropy = 0.491366
Epoch 39
Loss = 6.4089e-01, PNorm = 63.8817, GNorm = 2.5876, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 0.584887
Epoch 40
Validation binary_cross_entropy = 0.602236
Epoch 41
Validation binary_cross_entropy = 0.543659
Epoch 42
Validation binary_cross_entropy = 0.472970
Epoch 43
Validation binary_cross_entropy = 0.478900
Epoch 44
Loss = 6.5011e-01, PNorm = 63.9424, GNorm = 2.2508, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 0.502020
Epoch 45
Validation binary_cross_entropy = 0.531290
Epoch 46
Validation binary_cross_entropy = 0.605189
Epoch 47
Validation binary_cross_entropy = 0.578180
Epoch 48
Validation binary_cross_entropy = 0.472469
Epoch 49
Loss = 5.6602e-01, PNorm = 64.0065, GNorm = 2.1797, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 0.445336
Epoch 50
Validation binary_cross_entropy = 0.601404
Epoch 51
Validation binary_cross_entropy = 0.527437
Epoch 52
Validation binary_cross_entropy = 0.444580
Epoch 53
Validation binary_cross_entropy = 0.518978
Epoch 54
Loss = 5.0561e-01, PNorm = 64.0863, GNorm = 2.3914, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 0.513229
Epoch 55
Validation binary_cross_entropy = 0.416871
Epoch 56
Validation binary_cross_entropy = 0.385323
Epoch 57
Validation binary_cross_entropy = 0.439685
Epoch 58
Validation binary_cross_entropy = 0.477131
Epoch 59
Loss = 5.2129e-01, PNorm = 64.1718, GNorm = 1.8437, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 0.375356
Epoch 60
Validation binary_cross_entropy = 0.405288
Epoch 61
Validation binary_cross_entropy = 0.500979
Epoch 62
Validation binary_cross_entropy = 0.373644
Epoch 63
Validation binary_cross_entropy = 0.492720
Epoch 64
Loss = 4.4872e-01, PNorm = 64.2654, GNorm = 1.8776, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 0.475682
Epoch 65
Validation binary_cross_entropy = 0.407719
Epoch 66
Validation binary_cross_entropy = 0.432721
Epoch 67
Validation binary_cross_entropy = 0.347950
Epoch 68
Validation binary_cross_entropy = 0.510376
Epoch 69
Loss = 5.1953e-01, PNorm = 64.3588, GNorm = 1.8400, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 0.306464
Epoch 70
Validation binary_cross_entropy = 0.471980
Epoch 71
Validation binary_cross_entropy = 0.507622
Epoch 72
Validation binary_cross_entropy = 0.364043
Epoch 73
Validation binary_cross_entropy = 0.382895
Epoch 74
Loss = 4.3330e-01, PNorm = 64.4643, GNorm = 2.4359, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 0.396784
Epoch 75
Validation binary_cross_entropy = 0.327292
Epoch 76
Validation binary_cross_entropy = 0.387187
Epoch 77
Validation binary_cross_entropy = 0.335374
Epoch 78
Validation binary_cross_entropy = 0.425642
Epoch 79
Loss = 4.3273e-01, PNorm = 64.5663, GNorm = 1.8838, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 0.455353
Epoch 80
Validation binary_cross_entropy = 0.370196
Epoch 81
Validation binary_cross_entropy = 0.391624
Epoch 82
Validation binary_cross_entropy = 0.372607
Epoch 83
Validation binary_cross_entropy = 0.331110
Epoch 84
Loss = 3.4628e-01, PNorm = 64.6883, GNorm = 1.7338, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 0.508530
Epoch 85
Validation binary_cross_entropy = 0.338244
Epoch 86
Validation binary_cross_entropy = 0.408943
Epoch 87
Validation binary_cross_entropy = 0.457768
Epoch 88
Validation binary_cross_entropy = 0.318494
Epoch 89
Loss = 4.5395e-01, PNorm = 64.8013, GNorm = 2.4420, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 0.482693
Epoch 90
Validation binary_cross_entropy = 0.375504
Epoch 91
Validation binary_cross_entropy = 0.310761
Epoch 92
Validation binary_cross_entropy = 0.521798
Epoch 93
Validation binary_cross_entropy = 0.321695
Epoch 94
Loss = 4.8637e-01, PNorm = 64.9364, GNorm = 4.4356, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 0.308631
Epoch 95
Validation binary_cross_entropy = 0.458235
Epoch 96
Validation binary_cross_entropy = 0.425203
Epoch 97
Validation binary_cross_entropy = 0.341972
Epoch 98
Validation binary_cross_entropy = 0.410270
Epoch 99
Loss = 3.4311e-01, PNorm = 65.0810, GNorm = 1.9150, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.517458
Epoch 100
Validation binary_cross_entropy = 0.347972
Epoch 101
Validation binary_cross_entropy = 0.395681
Epoch 102
Validation binary_cross_entropy = 0.401147
Epoch 103
Validation binary_cross_entropy = 0.346039
Epoch 104
Loss = 2.8668e-01, PNorm = 65.2046, GNorm = 1.5033, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.396467
Epoch 105
Validation binary_cross_entropy = 0.425795
Epoch 106
Validation binary_cross_entropy = 0.441654
Epoch 107
Validation binary_cross_entropy = 0.519421
Epoch 108
Validation binary_cross_entropy = 0.465037
Epoch 109
Loss = 2.7748e-01, PNorm = 65.3302, GNorm = 2.1324, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.522512
Epoch 110
Validation binary_cross_entropy = 0.335623
Epoch 111
Validation binary_cross_entropy = 0.290490
Epoch 112
Validation binary_cross_entropy = 0.365171
Epoch 113
Validation binary_cross_entropy = 0.524182
Epoch 114
Loss = 5.0223e-01, PNorm = 65.4578, GNorm = 1.6733, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.336200
Epoch 115
Validation binary_cross_entropy = 0.297966
Epoch 116
Validation binary_cross_entropy = 0.339121
Epoch 117
Validation binary_cross_entropy = 0.435102
Epoch 118
Validation binary_cross_entropy = 0.436798
Epoch 119
Loss = 2.9040e-01, PNorm = 65.5984, GNorm = 2.1362, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.489594
Epoch 120
Validation binary_cross_entropy = 0.499471
Epoch 121
Validation binary_cross_entropy = 0.494762
Epoch 122
Validation binary_cross_entropy = 0.456593
Epoch 123
Validation binary_cross_entropy = 0.440197
Epoch 124
Loss = 2.0595e-01, PNorm = 65.7118, GNorm = 1.5369, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.405393
Epoch 125
Validation binary_cross_entropy = 0.411530
Epoch 126
Validation binary_cross_entropy = 0.423887
Epoch 127
Validation binary_cross_entropy = 0.446198
Epoch 128
Validation binary_cross_entropy = 0.481814
Epoch 129
Loss = 1.4175e-01, PNorm = 65.8289, GNorm = 1.7824, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.521217
Epoch 130
Validation binary_cross_entropy = 0.537195
Epoch 131
Validation binary_cross_entropy = 0.512177
Epoch 132
Validation binary_cross_entropy = 0.504164
Epoch 133
Validation binary_cross_entropy = 0.484348
Epoch 134
Loss = 1.5122e-01, PNorm = 65.9322, GNorm = 2.6983, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.498956
Epoch 135
Validation binary_cross_entropy = 0.435979
Epoch 136
Validation binary_cross_entropy = 0.430129
Epoch 137
Validation binary_cross_entropy = 0.484654
Epoch 138
Validation binary_cross_entropy = 0.455105
Epoch 139
Loss = 1.8520e-01, PNorm = 66.0361, GNorm = 5.7245, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.463649
Epoch 140
Validation binary_cross_entropy = 0.518297
Epoch 141
Validation binary_cross_entropy = 0.485850
Epoch 142
Validation binary_cross_entropy = 0.527778
Epoch 143
Validation binary_cross_entropy = 0.513994
Epoch 144
Loss = 1.9564e-01, PNorm = 66.1431, GNorm = 2.5383, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.617826
Epoch 145
Validation binary_cross_entropy = 0.521381
Epoch 146
Validation binary_cross_entropy = 0.532412
Epoch 147
Validation binary_cross_entropy = 0.519980
Epoch 148
Validation binary_cross_entropy = 0.495770
Epoch 149
Loss = 1.7205e-01, PNorm = 66.2447, GNorm = 0.9299, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.506710
Epoch 150
Validation binary_cross_entropy = 0.531751
Epoch 151
Validation binary_cross_entropy = 0.569683
Epoch 152
Validation binary_cross_entropy = 0.585215
Epoch 153
Validation binary_cross_entropy = 0.612885
Epoch 154
Loss = 1.2968e-01, PNorm = 66.3352, GNorm = 2.6267, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.620807
Epoch 155
Validation binary_cross_entropy = 0.623851
Epoch 156
Validation binary_cross_entropy = 0.637385
Epoch 157
Validation binary_cross_entropy = 0.650803
Epoch 158
Validation binary_cross_entropy = 0.676008
Epoch 159
Loss = 1.1395e-01, PNorm = 66.4153, GNorm = 1.3483, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.712153
Epoch 160
Validation binary_cross_entropy = 0.715740
Epoch 161
Validation binary_cross_entropy = 0.700757
Epoch 162
Validation binary_cross_entropy = 0.693427
Epoch 163
Validation binary_cross_entropy = 0.689762
Epoch 164
Loss = 1.3241e-01, PNorm = 66.4816, GNorm = 1.3883, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.665492
Epoch 165
Validation binary_cross_entropy = 0.640673
Epoch 166
Validation binary_cross_entropy = 0.649005
Epoch 167
Validation binary_cross_entropy = 0.670653
Epoch 168
Validation binary_cross_entropy = 0.688505
Epoch 169
Loss = 1.4489e-01, PNorm = 66.5509, GNorm = 1.3800, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.697485
Epoch 170
Validation binary_cross_entropy = 0.699114
Epoch 171
Validation binary_cross_entropy = 0.713904
Epoch 172
Validation binary_cross_entropy = 0.726096
Epoch 173
Validation binary_cross_entropy = 0.746595
Epoch 174
Loss = 7.4059e-02, PNorm = 66.6248, GNorm = 0.9997, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 0.812994
Epoch 175
Validation binary_cross_entropy = 0.866543
Epoch 176
Validation binary_cross_entropy = 0.880532
Epoch 177
Validation binary_cross_entropy = 0.920976
Epoch 178
Validation binary_cross_entropy = 0.941716
Epoch 179
Loss = 9.5584e-02, PNorm = 66.6998, GNorm = 1.1224, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 0.953082
Epoch 180
Validation binary_cross_entropy = 1.002095
Epoch 181
Validation binary_cross_entropy = 0.979125
Epoch 182
Validation binary_cross_entropy = 0.886137
Epoch 183
Validation binary_cross_entropy = 0.795813
Epoch 184
Loss = 9.4787e-02, PNorm = 66.7664, GNorm = 3.9095, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 0.780428
Epoch 185
Validation binary_cross_entropy = 0.774053
Epoch 186
Validation binary_cross_entropy = 0.747493
Epoch 187
Validation binary_cross_entropy = 0.785110
Epoch 188
Validation binary_cross_entropy = 0.797144
Epoch 189
Loss = 1.6604e-01, PNorm = 66.8575, GNorm = 1.0933, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 0.812701
Epoch 190
Validation binary_cross_entropy = 0.793171
Epoch 191
Validation binary_cross_entropy = 0.781198
Epoch 192
Validation binary_cross_entropy = 0.787704
Epoch 193
Validation binary_cross_entropy = 0.864741
Epoch 194
Loss = 1.0508e-01, PNorm = 66.9352, GNorm = 1.1071, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 0.841284
Epoch 195
Validation binary_cross_entropy = 0.828268
Epoch 196
Validation binary_cross_entropy = 0.821455
Epoch 197
Validation binary_cross_entropy = 0.849786
Epoch 198
Validation binary_cross_entropy = 0.828047
Epoch 199
Loss = 1.5126e-01, PNorm = 66.9967, GNorm = 1.6932, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 0.860740
Epoch 200
Validation binary_cross_entropy = 0.840062
Epoch 201
Validation binary_cross_entropy = 0.823786
Epoch 202
Validation binary_cross_entropy = 0.836618
Epoch 203
Validation binary_cross_entropy = 0.847698
Epoch 204
Loss = 9.2348e-02, PNorm = 67.0542, GNorm = 3.7293, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 0.810633
Epoch 205
Validation binary_cross_entropy = 0.788968
Epoch 206
Validation binary_cross_entropy = 0.811492
Epoch 207
Validation binary_cross_entropy = 0.827364
Epoch 208
Validation binary_cross_entropy = 0.896396
Epoch 209
Loss = 5.1363e-02, PNorm = 67.1177, GNorm = 1.5731, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 0.979551
Epoch 210
Validation binary_cross_entropy = 0.998246
Epoch 211
Validation binary_cross_entropy = 1.003611
Epoch 212
Validation binary_cross_entropy = 1.023584
Epoch 213
Validation binary_cross_entropy = 0.996808
Epoch 214
Loss = 3.8779e-02, PNorm = 67.1723, GNorm = 1.7164, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 0.994138
Epoch 215
Validation binary_cross_entropy = 0.992382
Epoch 216
Validation binary_cross_entropy = 1.032755
Epoch 217
Validation binary_cross_entropy = 1.046893
Epoch 218
Validation binary_cross_entropy = 1.087404
Epoch 219
Loss = 4.1366e-02, PNorm = 67.2220, GNorm = 0.8613, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 1.198201
Epoch 220
Validation binary_cross_entropy = 1.146503
Epoch 221
Validation binary_cross_entropy = 1.096649
Epoch 222
Validation binary_cross_entropy = 1.004526
Epoch 223
Validation binary_cross_entropy = 1.060539
Epoch 224
Loss = 1.5561e-01, PNorm = 67.2725, GNorm = 5.9531, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 0.984987
Epoch 225
Validation binary_cross_entropy = 0.860323
Epoch 226
Validation binary_cross_entropy = 0.796541
Epoch 227
Validation binary_cross_entropy = 0.799652
Epoch 228
Validation binary_cross_entropy = 0.830485
Epoch 229
Loss = 6.5051e-02, PNorm = 67.3385, GNorm = 0.8570, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 0.756934
Epoch 230
Validation binary_cross_entropy = 0.734597
Epoch 231
Validation binary_cross_entropy = 0.750151
Epoch 232
Validation binary_cross_entropy = 0.853515
Epoch 233
Validation binary_cross_entropy = 0.899295
Epoch 234
Loss = 5.9383e-02, PNorm = 67.4004, GNorm = 2.1420, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 0.889633
Epoch 235
Validation binary_cross_entropy = 0.933429
Epoch 236
Validation binary_cross_entropy = 0.978365
Epoch 237
Validation binary_cross_entropy = 1.199200
Epoch 238
Validation binary_cross_entropy = 0.878247
Epoch 239
Loss = 1.9607e-01, PNorm = 67.4600, GNorm = 2.8993, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 0.767469
Epoch 240
Validation binary_cross_entropy = 0.590216
Epoch 241
Validation binary_cross_entropy = 0.523086
Epoch 242
Validation binary_cross_entropy = 0.492568
Epoch 243
Validation binary_cross_entropy = 0.475630
Epoch 244
Loss = 1.9466e-01, PNorm = 67.5315, GNorm = 1.4084, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 0.509687
Epoch 245
Validation binary_cross_entropy = 0.555092
Epoch 246
Validation binary_cross_entropy = 0.585116
Epoch 247
Validation binary_cross_entropy = 0.716313
Epoch 248
Validation binary_cross_entropy = 0.831888
Epoch 249
Loss = 1.8624e-01, PNorm = 67.6102, GNorm = 4.6866, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 0.799740
Epoch 250
Validation binary_cross_entropy = 0.766069
Epoch 251
Validation binary_cross_entropy = 0.798226
Epoch 252
Validation binary_cross_entropy = 0.828309
Epoch 253
Validation binary_cross_entropy = 0.880801
Epoch 254
Loss = 7.7348e-02, PNorm = 67.6752, GNorm = 2.4505, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 0.941758
Epoch 255
Validation binary_cross_entropy = 0.903260
Epoch 256
Validation binary_cross_entropy = 0.894290
Epoch 257
Validation binary_cross_entropy = 0.906302
Epoch 258
Validation binary_cross_entropy = 0.935907
Epoch 259
Loss = 6.8140e-02, PNorm = 67.7337, GNorm = 1.5150, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 1.007627
Epoch 260
Validation binary_cross_entropy = 0.970751
Epoch 261
Validation binary_cross_entropy = 0.899228
Epoch 262
Validation binary_cross_entropy = 0.891010
Epoch 263
Validation binary_cross_entropy = 0.901733
Epoch 264
Loss = 8.8919e-02, PNorm = 67.7765, GNorm = 1.5320, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 0.913651
Epoch 265
Validation binary_cross_entropy = 0.864576
Epoch 266
Validation binary_cross_entropy = 0.867519
Epoch 267
Validation binary_cross_entropy = 0.886764
Epoch 268
Validation binary_cross_entropy = 0.912198
Epoch 269
Loss = 4.6500e-02, PNorm = 67.8294, GNorm = 0.6394, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 0.983108
Epoch 270
Validation binary_cross_entropy = 1.062547
Epoch 271
Validation binary_cross_entropy = 1.063928
Epoch 272
Validation binary_cross_entropy = 1.075137
Epoch 273
Validation binary_cross_entropy = 1.145001
Epoch 274
Loss = 8.5132e-02, PNorm = 67.8737, GNorm = 1.5733, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 1.094972
Epoch 275
Validation binary_cross_entropy = 1.053515
Epoch 276
Validation binary_cross_entropy = 1.049752
Epoch 277
Validation binary_cross_entropy = 1.052662
Epoch 278
Validation binary_cross_entropy = 1.061819
Epoch 279
Loss = 1.4937e-02, PNorm = 67.9177, GNorm = 0.3675, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 1.079000
Epoch 280
Validation binary_cross_entropy = 1.128458
Epoch 281
Validation binary_cross_entropy = 1.166914
Epoch 282
Validation binary_cross_entropy = 1.153983
Epoch 283
Validation binary_cross_entropy = 1.138781
Epoch 284
Loss = 5.3274e-02, PNorm = 67.9481, GNorm = 1.3720, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 1.133167
Epoch 285
Validation binary_cross_entropy = 1.154596
Epoch 286
Validation binary_cross_entropy = 1.093543
Epoch 287
Validation binary_cross_entropy = 1.023464
Epoch 288
Validation binary_cross_entropy = 1.006403
Epoch 289
Loss = 2.4342e-02, PNorm = 67.9829, GNorm = 1.6654, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 1.008850
Epoch 290
Validation binary_cross_entropy = 1.018823
Epoch 291
Validation binary_cross_entropy = 1.037742
Epoch 292
Validation binary_cross_entropy = 1.036773
Epoch 293
Validation binary_cross_entropy = 1.043579
Epoch 294
Loss = 7.0212e-02, PNorm = 68.0159, GNorm = 1.0527, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 1.067984
Epoch 295
Validation binary_cross_entropy = 1.085332
Epoch 296
Validation binary_cross_entropy = 1.102294
Epoch 297
Validation binary_cross_entropy = 1.114647
Epoch 298
Validation binary_cross_entropy = 1.204197
Epoch 299
Loss = 5.7316e-02, PNorm = 68.0507, GNorm = 3.8868, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 1.227961
Model 0 best validation binary_cross_entropy = 0.287546 on epoch 0
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.225327
Ensemble test binary_cross_entropy = 0.225327
Fold 9
Splitting data with seed 9
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.4, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=800, bias=False)
        (W_h): Linear(in_features=800, out_features=800, bias=False)
        (W_o): Linear(in_features=933, out_features=800, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.4, inplace=False)
    (1): Linear(in_features=3100, out_features=800, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.4, inplace=False)
    (4): Linear(in_features=800, out_features=800, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.4, inplace=False)
    (7): Linear(in_features=800, out_features=1, bias=True)
  )
)
Number of parameters = 4,627,201
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 0.288900
Epoch 1
Validation binary_cross_entropy = 0.296113
Epoch 2
Validation binary_cross_entropy = 0.627527
Epoch 3
Validation binary_cross_entropy = 0.325645
Epoch 4
Loss = 8.3044e-01, PNorm = 63.1499, GNorm = 6.1946, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 0.298872
Epoch 5
Validation binary_cross_entropy = 0.414138
Epoch 6
Validation binary_cross_entropy = 0.989933
Epoch 7
Validation binary_cross_entropy = 0.306474
Epoch 8
Validation binary_cross_entropy = 0.312818
Epoch 9
Loss = 6.8364e-01, PNorm = 63.2825, GNorm = 5.0200, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 0.934482
Epoch 10
Validation binary_cross_entropy = 0.547546
Epoch 11
Validation binary_cross_entropy = 0.309192
Epoch 12
Validation binary_cross_entropy = 0.598920
Epoch 13
Validation binary_cross_entropy = 0.757241
Epoch 14
Loss = 7.5174e-01, PNorm = 63.4329, GNorm = 4.7699, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 0.528832
Epoch 15
Validation binary_cross_entropy = 0.518389
Epoch 16
Validation binary_cross_entropy = 0.456228
Epoch 17
Validation binary_cross_entropy = 0.651107
Epoch 18
Validation binary_cross_entropy = 0.637769
Epoch 19
Loss = 7.0490e-01, PNorm = 63.5508, GNorm = 4.1631, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 0.411958
Epoch 20
Validation binary_cross_entropy = 0.379109
Epoch 21
Validation binary_cross_entropy = 0.663031
Epoch 22
Validation binary_cross_entropy = 0.654711
Epoch 23
Validation binary_cross_entropy = 0.513110
Epoch 24
Loss = 6.4934e-01, PNorm = 63.6422, GNorm = 4.8818, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 0.425396
Epoch 25
Validation binary_cross_entropy = 0.566951
Epoch 26
Validation binary_cross_entropy = 0.563345
Epoch 27
Validation binary_cross_entropy = 0.511907
Epoch 28
Validation binary_cross_entropy = 0.524909
Epoch 29
Loss = 5.9170e-01, PNorm = 63.7176, GNorm = 3.1235, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 0.557497
Epoch 30
Validation binary_cross_entropy = 0.508854
Epoch 31
Validation binary_cross_entropy = 0.505205
Epoch 32
Validation binary_cross_entropy = 0.682773
Epoch 33
Validation binary_cross_entropy = 0.551893
Epoch 34
Loss = 5.6338e-01, PNorm = 63.7647, GNorm = 2.2943, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 0.402073
Epoch 35
Validation binary_cross_entropy = 0.421800
Epoch 36
Validation binary_cross_entropy = 0.516861
Epoch 37
Validation binary_cross_entropy = 0.551832
Epoch 38
Validation binary_cross_entropy = 0.451983
Epoch 39
Loss = 7.0629e-01, PNorm = 63.8176, GNorm = 2.3840, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 0.445434
Epoch 40
Validation binary_cross_entropy = 0.500389
Epoch 41
Validation binary_cross_entropy = 0.567812
Epoch 42
Validation binary_cross_entropy = 0.546756
Epoch 43
Validation binary_cross_entropy = 0.509498
Epoch 44
Loss = 6.4506e-01, PNorm = 63.8690, GNorm = 2.4382, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 0.499064
Epoch 45
Validation binary_cross_entropy = 0.532511
Epoch 46
Validation binary_cross_entropy = 0.562847
Epoch 47
Validation binary_cross_entropy = 0.552566
Epoch 48
Validation binary_cross_entropy = 0.488790
Epoch 49
Loss = 6.0301e-01, PNorm = 63.9332, GNorm = 2.8309, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 0.509063
Epoch 50
Validation binary_cross_entropy = 0.425102
Epoch 51
Validation binary_cross_entropy = 0.462721
Epoch 52
Validation binary_cross_entropy = 0.551021
Epoch 53
Validation binary_cross_entropy = 0.589781
Epoch 54
Loss = 6.1229e-01, PNorm = 63.9844, GNorm = 2.2209, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 0.512728
Epoch 55
Validation binary_cross_entropy = 0.450623
Epoch 56
Validation binary_cross_entropy = 0.463310
Epoch 57
Validation binary_cross_entropy = 0.459335
Epoch 58
Validation binary_cross_entropy = 0.474010
Epoch 59
Loss = 5.5366e-01, PNorm = 64.0474, GNorm = 1.6926, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 0.503369
Epoch 60
Validation binary_cross_entropy = 0.505439
Epoch 61
Validation binary_cross_entropy = 0.500602
Epoch 62
Validation binary_cross_entropy = 0.497277
Epoch 63
Validation binary_cross_entropy = 0.481739
Epoch 64
Loss = 6.1077e-01, PNorm = 64.1116, GNorm = 1.5692, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 0.490339
Epoch 65
Validation binary_cross_entropy = 0.506147
Epoch 66
Validation binary_cross_entropy = 0.568908
Epoch 67
Validation binary_cross_entropy = 0.517063
Epoch 68
Validation binary_cross_entropy = 0.460587
Epoch 69
Loss = 5.2219e-01, PNorm = 64.1851, GNorm = 1.6659, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 0.461268
Epoch 70
Validation binary_cross_entropy = 0.541916
Epoch 71
Validation binary_cross_entropy = 0.515901
Epoch 72
Validation binary_cross_entropy = 0.422997
Epoch 73
Validation binary_cross_entropy = 0.424317
Epoch 74
Loss = 4.9852e-01, PNorm = 64.2593, GNorm = 2.2279, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 0.695956
Epoch 75
Validation binary_cross_entropy = 0.474141
Epoch 76
Validation binary_cross_entropy = 0.380577
Epoch 77
Validation binary_cross_entropy = 0.509671
Epoch 78
Validation binary_cross_entropy = 0.649239
Epoch 79
Loss = 5.6326e-01, PNorm = 64.3305, GNorm = 2.1418, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 0.601090
Epoch 80
Validation binary_cross_entropy = 0.461573
Epoch 81
Validation binary_cross_entropy = 0.446582
Epoch 82
Validation binary_cross_entropy = 0.552948
Epoch 83
Validation binary_cross_entropy = 0.568176
Epoch 84
Loss = 5.2070e-01, PNorm = 64.4223, GNorm = 1.5745, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 0.425082
Epoch 85
Validation binary_cross_entropy = 0.661683
Epoch 86
Validation binary_cross_entropy = 0.649528
Epoch 87
Validation binary_cross_entropy = 0.370485
Epoch 88
Validation binary_cross_entropy = 0.467967
Epoch 89
Loss = 5.2102e-01, PNorm = 64.5004, GNorm = 1.8889, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 0.570789
Epoch 90
Validation binary_cross_entropy = 0.531688
Epoch 91
Validation binary_cross_entropy = 0.448440
Epoch 92
Validation binary_cross_entropy = 0.421496
Epoch 93
Validation binary_cross_entropy = 0.372979
Epoch 94
Loss = 4.9588e-01, PNorm = 64.5905, GNorm = 1.6387, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 0.417093
Epoch 95
Validation binary_cross_entropy = 0.459382
Epoch 96
Validation binary_cross_entropy = 0.401655
Epoch 97
Validation binary_cross_entropy = 0.400023
Epoch 98
Validation binary_cross_entropy = 0.435690
Epoch 99
Loss = 6.7552e-01, PNorm = 64.6970, GNorm = 6.3244, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.360922
Epoch 100
Validation binary_cross_entropy = 0.453231
Epoch 101
Validation binary_cross_entropy = 0.692516
Epoch 102
Validation binary_cross_entropy = 0.513603
Epoch 103
Validation binary_cross_entropy = 0.398976
Epoch 104
Loss = 4.7763e-01, PNorm = 64.7862, GNorm = 1.3288, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.360552
Epoch 105
Validation binary_cross_entropy = 0.414690
Epoch 106
Validation binary_cross_entropy = 0.504618
Epoch 107
Validation binary_cross_entropy = 0.561164
Epoch 108
Validation binary_cross_entropy = 0.480846
Epoch 109
Loss = 4.2861e-01, PNorm = 64.8959, GNorm = 2.3231, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.404064
Epoch 110
Validation binary_cross_entropy = 0.435686
Epoch 111
Validation binary_cross_entropy = 0.449121
Epoch 112
Validation binary_cross_entropy = 0.405311
Epoch 113
Validation binary_cross_entropy = 0.410347
Epoch 114
Loss = 5.1305e-01, PNorm = 64.9679, GNorm = 1.6515, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.447559
Epoch 115
Validation binary_cross_entropy = 0.422570
Epoch 116
Validation binary_cross_entropy = 0.444012
Epoch 117
Validation binary_cross_entropy = 0.525262
Epoch 118
Validation binary_cross_entropy = 0.411156
Epoch 119
Loss = 4.0218e-01, PNorm = 65.0634, GNorm = 1.5317, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.405639
Epoch 120
Validation binary_cross_entropy = 0.410765
Epoch 121
Validation binary_cross_entropy = 0.427659
Epoch 122
Validation binary_cross_entropy = 0.405211
Epoch 123
Validation binary_cross_entropy = 0.442911
Epoch 124
Loss = 3.0890e-01, PNorm = 65.1671, GNorm = 1.6124, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.394777
Epoch 125
Validation binary_cross_entropy = 0.590893
Epoch 126
Validation binary_cross_entropy = 0.406036
Epoch 127
Validation binary_cross_entropy = 0.428446
Epoch 128
Validation binary_cross_entropy = 0.475835
Epoch 129
Loss = 2.7375e-01, PNorm = 65.2914, GNorm = 1.6055, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.474203
Epoch 130
Validation binary_cross_entropy = 0.435007
Epoch 131
Validation binary_cross_entropy = 0.577607
Epoch 132
Validation binary_cross_entropy = 0.355320
Epoch 133
Validation binary_cross_entropy = 0.376572
Epoch 134
Loss = 3.5452e-01, PNorm = 65.4069, GNorm = 3.2060, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.556744
Epoch 135
Validation binary_cross_entropy = 0.391409
Epoch 136
Validation binary_cross_entropy = 0.340950
Epoch 137
Validation binary_cross_entropy = 0.413061
Epoch 138
Validation binary_cross_entropy = 0.511444
Epoch 139
Loss = 2.1878e-01, PNorm = 65.5293, GNorm = 1.8538, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.610915
Epoch 140
Validation binary_cross_entropy = 0.571178
Epoch 141
Validation binary_cross_entropy = 0.543512
Epoch 142
Validation binary_cross_entropy = 0.520974
Epoch 143
Validation binary_cross_entropy = 0.574298
Epoch 144
Loss = 3.0774e-01, PNorm = 65.6556, GNorm = 2.9348, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.389428
Epoch 145
Validation binary_cross_entropy = 0.369535
Epoch 146
Validation binary_cross_entropy = 0.412468
Epoch 147
Validation binary_cross_entropy = 0.516225
Epoch 148
Validation binary_cross_entropy = 0.455079
Epoch 149
Loss = 2.4803e-01, PNorm = 65.7841, GNorm = 2.4804, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.451709
Epoch 150
Validation binary_cross_entropy = 0.481297
Epoch 151
Validation binary_cross_entropy = 0.581413
Epoch 152
Validation binary_cross_entropy = 0.496244
Epoch 153
Validation binary_cross_entropy = 0.536737
Epoch 154
Loss = 2.9935e-01, PNorm = 65.9096, GNorm = 2.0373, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.504525
Epoch 155
Validation binary_cross_entropy = 0.532952
Epoch 156
Validation binary_cross_entropy = 0.568003
Epoch 157
Validation binary_cross_entropy = 0.622010
Epoch 158
Validation binary_cross_entropy = 0.691760
Epoch 159
Loss = 2.1128e-01, PNorm = 66.0280, GNorm = 2.7696, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.680182
Epoch 160
Validation binary_cross_entropy = 0.674138
Epoch 161
Validation binary_cross_entropy = 0.728066
Epoch 162
Validation binary_cross_entropy = 0.690325
Epoch 163
Validation binary_cross_entropy = 0.691302
Epoch 164
Loss = 2.6650e-01, PNorm = 66.1355, GNorm = 3.8853, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.615440
Epoch 165
Validation binary_cross_entropy = 0.651972
Epoch 166
Validation binary_cross_entropy = 0.674294
Epoch 167
Validation binary_cross_entropy = 0.675057
Epoch 168
Validation binary_cross_entropy = 0.655449
Epoch 169
Loss = 2.4711e-01, PNorm = 66.2601, GNorm = 3.1258, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.625582
Epoch 170
Validation binary_cross_entropy = 0.685914
Epoch 171
Validation binary_cross_entropy = 0.578064
Epoch 172
Validation binary_cross_entropy = 0.594077
Epoch 173
Validation binary_cross_entropy = 0.608796
Epoch 174
Loss = 2.2821e-01, PNorm = 66.3676, GNorm = 1.6336, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 0.612264
Epoch 175
Validation binary_cross_entropy = 0.622833
Epoch 176
Validation binary_cross_entropy = 0.655566
Epoch 177
Validation binary_cross_entropy = 0.730178
Epoch 178
Validation binary_cross_entropy = 0.675535
Epoch 179
Loss = 2.1640e-01, PNorm = 66.4821, GNorm = 3.0202, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 0.635750
Epoch 180
Validation binary_cross_entropy = 0.631667
Epoch 181
Validation binary_cross_entropy = 0.655540
Epoch 182
Validation binary_cross_entropy = 0.664697
Epoch 183
Validation binary_cross_entropy = 0.668432
Epoch 184
Loss = 1.1927e-01, PNorm = 66.5751, GNorm = 1.7089, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 0.679258
Epoch 185
Validation binary_cross_entropy = 0.718565
Epoch 186
Validation binary_cross_entropy = 0.710256
Epoch 187
Validation binary_cross_entropy = 0.765935
Epoch 188
Validation binary_cross_entropy = 0.761811
Epoch 189
Loss = 1.4232e-01, PNorm = 66.6724, GNorm = 1.1739, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 0.659185
Epoch 190
Validation binary_cross_entropy = 0.652364
Epoch 191
Validation binary_cross_entropy = 0.806517
Epoch 192
Validation binary_cross_entropy = 0.730548
Epoch 193
Validation binary_cross_entropy = 0.758978
Epoch 194
Loss = 1.6751e-01, PNorm = 66.7763, GNorm = 3.7780, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 0.785842
Epoch 195
Validation binary_cross_entropy = 0.761353
Epoch 196
Validation binary_cross_entropy = 0.827404
Epoch 197
Validation binary_cross_entropy = 0.715253
Epoch 198
Validation binary_cross_entropy = 0.685623
Epoch 199
Loss = 1.3429e-01, PNorm = 66.8715, GNorm = 2.4346, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 0.727729
Epoch 200
Validation binary_cross_entropy = 0.787337
Epoch 201
Validation binary_cross_entropy = 0.888449
Epoch 202
Validation binary_cross_entropy = 0.862079
Epoch 203
Validation binary_cross_entropy = 0.858547
Epoch 204
Loss = 2.3100e-01, PNorm = 66.9509, GNorm = 1.2725, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 0.952700
Epoch 205
Validation binary_cross_entropy = 0.940463
Epoch 206
Validation binary_cross_entropy = 0.833851
Epoch 207
Validation binary_cross_entropy = 0.856566
Epoch 208
Validation binary_cross_entropy = 0.805459
Epoch 209
Loss = 4.8214e-02, PNorm = 67.0231, GNorm = 0.6904, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 0.947593
Epoch 210
Validation binary_cross_entropy = 0.856513
Epoch 211
Validation binary_cross_entropy = 0.780611
Epoch 212
Validation binary_cross_entropy = 0.789957
Epoch 213
Validation binary_cross_entropy = 0.844410
Epoch 214
Loss = 9.6699e-02, PNorm = 67.0992, GNorm = 1.8094, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 0.959258
Epoch 215
Validation binary_cross_entropy = 0.997409
Epoch 216
Validation binary_cross_entropy = 1.025235
Epoch 217
Validation binary_cross_entropy = 1.052710
Epoch 218
Validation binary_cross_entropy = 1.092445
Epoch 219
Loss = 7.7272e-02, PNorm = 67.1685, GNorm = 0.8569, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 1.139068
Epoch 220
Validation binary_cross_entropy = 1.094252
Epoch 221
Validation binary_cross_entropy = 1.032983
Epoch 222
Validation binary_cross_entropy = 1.005158
Epoch 223
Validation binary_cross_entropy = 0.996140
Epoch 224
Loss = 1.1100e-01, PNorm = 67.2390, GNorm = 0.7395, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 1.004528
Epoch 225
Validation binary_cross_entropy = 1.010194
Epoch 226
Validation binary_cross_entropy = 1.001788
Epoch 227
Validation binary_cross_entropy = 1.081595
Epoch 228
Validation binary_cross_entropy = 1.133931
Epoch 229
Loss = 1.4656e-01, PNorm = 67.3158, GNorm = 0.9273, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 0.923799
Epoch 230
Validation binary_cross_entropy = 0.903259
Epoch 231
Validation binary_cross_entropy = 0.943619
Epoch 232
Validation binary_cross_entropy = 0.983836
Epoch 233
Validation binary_cross_entropy = 0.760726
Epoch 234
Loss = 1.0202e-01, PNorm = 67.3863, GNorm = 2.3109, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 0.782940
Epoch 235
Validation binary_cross_entropy = 0.754580
Epoch 236
Validation binary_cross_entropy = 0.827625
Epoch 237
Validation binary_cross_entropy = 0.960367
Epoch 238
Validation binary_cross_entropy = 0.967375
Epoch 239
Loss = 9.3937e-02, PNorm = 67.4737, GNorm = 1.4101, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 0.976392
Epoch 240
Validation binary_cross_entropy = 1.004143
Epoch 241
Validation binary_cross_entropy = 1.125548
Epoch 242
Validation binary_cross_entropy = 1.304341
Epoch 243
Validation binary_cross_entropy = 1.015541
Epoch 244
Loss = 1.0096e-01, PNorm = 67.5400, GNorm = 0.7349, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 0.929505
Epoch 245
Validation binary_cross_entropy = 0.912798
Epoch 246
Validation binary_cross_entropy = 0.920925
Epoch 247
Validation binary_cross_entropy = 1.021226
Epoch 248
Validation binary_cross_entropy = 0.999889
Epoch 249
Loss = 5.1999e-02, PNorm = 67.6054, GNorm = 1.0720, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 0.903183
Epoch 250
Validation binary_cross_entropy = 0.923696
Epoch 251
Validation binary_cross_entropy = 1.000984
Epoch 252
Validation binary_cross_entropy = 0.945743
Epoch 253
Validation binary_cross_entropy = 0.879110
Epoch 254
Loss = 7.3290e-02, PNorm = 67.6626, GNorm = 0.6395, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 0.863368
Epoch 255
Validation binary_cross_entropy = 0.880684
Epoch 256
Validation binary_cross_entropy = 0.906490
Epoch 257
Validation binary_cross_entropy = 0.933370
Epoch 258
Validation binary_cross_entropy = 0.962841
Epoch 259
Loss = 5.8618e-02, PNorm = 67.7211, GNorm = 0.3700, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 1.029311
Epoch 260
Validation binary_cross_entropy = 1.102662
Epoch 261
Validation binary_cross_entropy = 1.198411
Epoch 262
Validation binary_cross_entropy = 1.314830
Epoch 263
Validation binary_cross_entropy = 1.335524
Epoch 264
Loss = 3.7244e-02, PNorm = 67.7764, GNorm = 0.2778, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 1.234284
Epoch 265
Validation binary_cross_entropy = 1.247617
Epoch 266
Validation binary_cross_entropy = 1.204413
Epoch 267
Validation binary_cross_entropy = 1.306379
Epoch 268
Validation binary_cross_entropy = 1.207617
Epoch 269
Loss = 5.7970e-02, PNorm = 67.8184, GNorm = 1.6239, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 1.115563
Epoch 270
Validation binary_cross_entropy = 1.126639
Epoch 271
Validation binary_cross_entropy = 1.175565
Epoch 272
Validation binary_cross_entropy = 1.278191
Epoch 273
Validation binary_cross_entropy = 1.281467
Epoch 274
Loss = 6.7799e-02, PNorm = 67.8727, GNorm = 1.2204, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 1.275515
Epoch 275
Validation binary_cross_entropy = 1.318951
Epoch 276
Validation binary_cross_entropy = 1.365902
Epoch 277
Validation binary_cross_entropy = 1.440852
Epoch 278
Validation binary_cross_entropy = 1.441641
Epoch 279
Loss = 5.7664e-02, PNorm = 67.9232, GNorm = 1.5768, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 1.499216
Epoch 280
Validation binary_cross_entropy = 1.490388
Epoch 281
Validation binary_cross_entropy = 1.520262
Epoch 282
Validation binary_cross_entropy = 1.535105
Epoch 283
Validation binary_cross_entropy = 1.727079
Epoch 284
Loss = 2.7786e-02, PNorm = 67.9711, GNorm = 2.1677, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 1.763739
Epoch 285
Validation binary_cross_entropy = 1.645660
Epoch 286
Validation binary_cross_entropy = 1.650072
Epoch 287
Validation binary_cross_entropy = 1.713339
Epoch 288
Validation binary_cross_entropy = 1.795808
Epoch 289
Loss = 1.5754e-02, PNorm = 68.0131, GNorm = 2.4261, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 1.862291
Epoch 290
Validation binary_cross_entropy = 1.805245
Epoch 291
Validation binary_cross_entropy = 1.762914
Epoch 292
Validation binary_cross_entropy = 1.975118
Epoch 293
Validation binary_cross_entropy = 2.003293
Epoch 294
Loss = 7.3219e-02, PNorm = 68.0479, GNorm = 3.3368, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 1.808892
Epoch 295
Validation binary_cross_entropy = 1.630189
Epoch 296
Validation binary_cross_entropy = 1.545518
Epoch 297
Validation binary_cross_entropy = 1.543763
Epoch 298
Validation binary_cross_entropy = 1.596632
Epoch 299
Loss = 9.9779e-03, PNorm = 68.0834, GNorm = 0.4457, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 1.636361
Model 0 best validation binary_cross_entropy = 0.288900 on epoch 0
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.222776
Ensemble test binary_cross_entropy = 0.222776
10-fold cross validation
	Seed 0 ==> test binary_cross_entropy = 0.242493
	Seed 1 ==> test binary_cross_entropy = 0.198337
	Seed 2 ==> test binary_cross_entropy = 0.205699
	Seed 3 ==> test binary_cross_entropy = 0.244754
	Seed 4 ==> test binary_cross_entropy = 0.193636
	Seed 5 ==> test binary_cross_entropy = 0.211122
	Seed 6 ==> test binary_cross_entropy = 0.227363
	Seed 7 ==> test binary_cross_entropy = 0.223729
	Seed 8 ==> test binary_cross_entropy = 0.225327
	Seed 9 ==> test binary_cross_entropy = 0.222776
Overall test binary_cross_entropy = 0.219524 +/- 0.016335
Elapsed time = 0:05:19
Command line
python /home/saxelrod/Repo/projects/chemprop/chemprop/train.py --config_path /home/saxelrod/chemprop_cov_2/models/from_cp_feats_mpnn_from_binary_cross_entropy/config.json --data_path /home/saxelrod/chemprop_cov_2/scaffold_split/train_full.csv --dataset_type classification
Args
{'activation': 'ReLU',
 'aggregation': 'mean',
 'aggregation_norm': 100,
 'atom_descriptors': None,
 'atom_descriptors_path': None,
 'atom_descriptors_size': 0,
 'atom_features_size': 0,
 'atom_messages': False,
 'batch_size': 50,
 'bias': False,
 'cache_cutoff': 10000,
 'checkpoint_dir': None,
 'checkpoint_path': None,
 'checkpoint_paths': None,
 'class_balance': True,
 'config_path': '/home/saxelrod/chemprop_cov_2/models/from_cp_feats_mpnn_from_binary_cross_entropy/config.json',
 'crossval_index_dir': None,
 'crossval_index_file': None,
 'crossval_index_sets': None,
 'cuda': True,
 'data_path': '/home/saxelrod/chemprop_cov_2/scaffold_split/train_full.csv',
 'dataset_type': 'classification',
 'depth': 6,
 'device': device(type='cuda', index=1),
 'dropout': 0.4,
 'ensemble_size': 1,
 'epochs': 300,
 'extra_metrics': [],
 'features_generator': None,
 'features_only': False,
 'features_path': ['/home/saxelrod/chemprop_cov_2/features/from_cp/train_binary_cross_entropy.npz'],
 'features_scaling': False,
 'features_size': None,
 'ffn_hidden_size': 800,
 'ffn_num_layers': 3,
 'final_lr': 0.0001,
 'folds_file': None,
 'gpu': 1,
 'grad_clip': None,
 'hidden_size': 800,
 'ignore_columns': None,
 'init_lr': 0.0001,
 'log_frequency': 10,
 'max_data_size': None,
 'max_lr': 0.001,
 'metric': 'binary_cross_entropy',
 'metrics': ['binary_cross_entropy'],
 'minimize_score': True,
 'mpn_shared': False,
 'multiclass_num_classes': 3,
 'no_cache_mol': False,
 'no_cuda': False,
 'no_features_scaling': True,
 'num_folds': 10,
 'num_lrs': 1,
 'num_tasks': 1,
 'num_workers': 8,
 'number_of_molecules': 1,
 'pytorch_seed': 0,
 'quiet': True,
 'save_dir': '/home/saxelrod/chemprop_cov_2/models/from_cp_feats_mpnn_from_binary_cross_entropy',
 'save_preds': False,
 'save_smiles_splits': False,
 'seed': 0,
 'separate_test_features_path': ['/home/saxelrod/chemprop_cov_2/features/from_cp/test_binary_cross_entropy.npz'],
 'separate_test_path': '/home/saxelrod/chemprop_cov_2/scaffold_split/test_full.csv',
 'separate_val_features_path': ['/home/saxelrod/chemprop_cov_2/features/from_cp/val_binary_cross_entropy.npz'],
 'separate_val_path': '/home/saxelrod/chemprop_cov_2/scaffold_split/val_full.csv',
 'show_individual_scores': False,
 'smiles_columns': [None],
 'split_sizes': (0.8, 0.1, 0.1),
 'split_type': 'random',
 'target_columns': None,
 'task_names': ['sars_cov_two_cl_protease_active'],
 'test': False,
 'test_fold_index': None,
 'train_data_size': None,
 'undirected': False,
 'use_input_features': True,
 'val_fold_index': None,
 'warmup_epochs': 2.0}
Loading data
Number of tasks = 1
Fold 0
Splitting data with seed 0
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.4, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=800, bias=False)
        (W_h): Linear(in_features=800, out_features=800, bias=False)
        (W_o): Linear(in_features=933, out_features=800, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.4, inplace=False)
    (1): Linear(in_features=3100, out_features=800, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.4, inplace=False)
    (4): Linear(in_features=800, out_features=800, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.4, inplace=False)
    (7): Linear(in_features=800, out_features=1, bias=True)
  )
)
Number of parameters = 4,627,201
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 0.291513
Epoch 1
Validation binary_cross_entropy = 0.300563
Epoch 2
Validation binary_cross_entropy = 0.549332
Epoch 3
Validation binary_cross_entropy = 0.331805
Epoch 4
Loss = 8.2302e-01, PNorm = 63.1518, GNorm = 7.3082, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 0.310001
Epoch 5
Validation binary_cross_entropy = 0.413272
Epoch 6
Validation binary_cross_entropy = 0.632496
Epoch 7
Validation binary_cross_entropy = 0.291358
Epoch 8
Validation binary_cross_entropy = 0.298074
Epoch 9
Loss = 7.6393e-01, PNorm = 63.2881, GNorm = 5.3321, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 0.638309
Epoch 10
Validation binary_cross_entropy = 0.450271
Epoch 11
Validation binary_cross_entropy = 0.313449
Epoch 12
Validation binary_cross_entropy = 0.491421
Epoch 13
Validation binary_cross_entropy = 0.476504
Epoch 14
Loss = 8.1098e-01, PNorm = 63.4439, GNorm = 4.9537, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 0.327053
Epoch 15
Validation binary_cross_entropy = 0.434943
Epoch 16
Validation binary_cross_entropy = 0.539258
Epoch 17
Validation binary_cross_entropy = 0.415537
Epoch 18
Validation binary_cross_entropy = 0.527824
Epoch 19
Loss = 6.9978e-01, PNorm = 63.5571, GNorm = 4.2303, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 0.408644
Epoch 20
Validation binary_cross_entropy = 0.369876
Epoch 21
Validation binary_cross_entropy = 0.589452
Epoch 22
Validation binary_cross_entropy = 0.481779
Epoch 23
Validation binary_cross_entropy = 0.485017
Epoch 24
Loss = 6.6369e-01, PNorm = 63.6398, GNorm = 2.8893, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 0.437743
Epoch 25
Validation binary_cross_entropy = 0.585645
Epoch 26
Validation binary_cross_entropy = 0.537080
Epoch 27
Validation binary_cross_entropy = 0.552320
Epoch 28
Validation binary_cross_entropy = 0.499481
Epoch 29
Loss = 5.9060e-01, PNorm = 63.7167, GNorm = 2.8534, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 0.480392
Epoch 30
Validation binary_cross_entropy = 0.443153
Epoch 31
Validation binary_cross_entropy = 0.530520
Epoch 32
Validation binary_cross_entropy = 0.685076
Epoch 33
Validation binary_cross_entropy = 0.514649
Epoch 34
Loss = 7.3160e-01, PNorm = 63.8002, GNorm = 5.0103, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 0.427233
Epoch 35
Validation binary_cross_entropy = 0.507239
Epoch 36
Validation binary_cross_entropy = 0.536830
Epoch 37
Validation binary_cross_entropy = 0.542240
Epoch 38
Validation binary_cross_entropy = 0.477961
Epoch 39
Loss = 5.9979e-01, PNorm = 63.8617, GNorm = 2.2587, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 0.490366
Epoch 40
Validation binary_cross_entropy = 0.501424
Epoch 41
Validation binary_cross_entropy = 0.493479
Epoch 42
Validation binary_cross_entropy = 0.478281
Epoch 43
Validation binary_cross_entropy = 0.497873
Epoch 44
Loss = 6.4639e-01, PNorm = 63.9125, GNorm = 2.9476, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 0.497061
Epoch 45
Validation binary_cross_entropy = 0.504444
Epoch 46
Validation binary_cross_entropy = 0.524794
Epoch 47
Validation binary_cross_entropy = 0.565214
Epoch 48
Validation binary_cross_entropy = 0.571733
Epoch 49
Loss = 5.7608e-01, PNorm = 63.9690, GNorm = 2.0715, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 0.566454
Epoch 50
Validation binary_cross_entropy = 0.489663
Epoch 51
Validation binary_cross_entropy = 0.466363
Epoch 52
Validation binary_cross_entropy = 0.527071
Epoch 53
Validation binary_cross_entropy = 0.585976
Epoch 54
Loss = 5.8842e-01, PNorm = 64.0205, GNorm = 1.6390, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 0.606649
Epoch 55
Validation binary_cross_entropy = 0.569462
Epoch 56
Validation binary_cross_entropy = 0.489244
Epoch 57
Validation binary_cross_entropy = 0.450669
Epoch 58
Validation binary_cross_entropy = 0.498745
Epoch 59
Loss = 5.4210e-01, PNorm = 64.0860, GNorm = 2.1165, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 0.594303
Epoch 60
Validation binary_cross_entropy = 0.565601
Epoch 61
Validation binary_cross_entropy = 0.526246
Epoch 62
Validation binary_cross_entropy = 0.553257
Epoch 63
Validation binary_cross_entropy = 0.365143
Epoch 64
Loss = 5.6708e-01, PNorm = 64.1581, GNorm = 3.8693, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 0.424702
Epoch 65
Validation binary_cross_entropy = 0.734818
Epoch 66
Validation binary_cross_entropy = 0.593481
Epoch 67
Validation binary_cross_entropy = 0.479614
Epoch 68
Validation binary_cross_entropy = 0.436375
Epoch 69
Loss = 4.8166e-01, PNorm = 64.2429, GNorm = 1.3508, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 0.496648
Epoch 70
Validation binary_cross_entropy = 0.448676
Epoch 71
Validation binary_cross_entropy = 0.368955
Epoch 72
Validation binary_cross_entropy = 0.416038
Epoch 73
Validation binary_cross_entropy = 0.405116
Epoch 74
Loss = 4.6212e-01, PNorm = 64.3381, GNorm = 1.7301, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 0.385286
Epoch 75
Validation binary_cross_entropy = 0.317430
Epoch 76
Validation binary_cross_entropy = 0.640763
Epoch 77
Validation binary_cross_entropy = 0.456983
Epoch 78
Validation binary_cross_entropy = 0.391593
Epoch 79
Loss = 5.0234e-01, PNorm = 64.4190, GNorm = 2.3067, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 0.424095
Epoch 80
Validation binary_cross_entropy = 0.564847
Epoch 81
Validation binary_cross_entropy = 0.729193
Epoch 82
Validation binary_cross_entropy = 0.543196
Epoch 83
Validation binary_cross_entropy = 0.400939
Epoch 84
Loss = 5.0515e-01, PNorm = 64.5331, GNorm = 2.1307, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 0.419389
Epoch 85
Validation binary_cross_entropy = 0.588215
Epoch 86
Validation binary_cross_entropy = 0.423592
Epoch 87
Validation binary_cross_entropy = 0.382869
Epoch 88
Validation binary_cross_entropy = 0.428927
Epoch 89
Loss = 4.7803e-01, PNorm = 64.6275, GNorm = 1.5439, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 0.550175
Epoch 90
Validation binary_cross_entropy = 0.486576
Epoch 91
Validation binary_cross_entropy = 0.395095
Epoch 92
Validation binary_cross_entropy = 0.368515
Epoch 93
Validation binary_cross_entropy = 0.364157
Epoch 94
Loss = 3.8211e-01, PNorm = 64.7266, GNorm = 1.6379, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 0.492058
Epoch 95
Validation binary_cross_entropy = 0.335302
Epoch 96
Validation binary_cross_entropy = 0.334527
Epoch 97
Validation binary_cross_entropy = 0.423339
Epoch 98
Validation binary_cross_entropy = 0.435534
Epoch 99
Loss = 4.0832e-01, PNorm = 64.8207, GNorm = 1.9100, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.332675
Epoch 100
Validation binary_cross_entropy = 0.340582
Epoch 101
Validation binary_cross_entropy = 0.458380
Epoch 102
Validation binary_cross_entropy = 0.325717
Epoch 103
Validation binary_cross_entropy = 0.721613
Epoch 104
Loss = 4.9358e-01, PNorm = 64.9307, GNorm = 3.6929, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.539421
Epoch 105
Validation binary_cross_entropy = 0.292342
Epoch 106
Validation binary_cross_entropy = 0.315638
Epoch 107
Validation binary_cross_entropy = 0.412613
Epoch 108
Validation binary_cross_entropy = 0.486160
Epoch 109
Loss = 4.4880e-01, PNorm = 65.0490, GNorm = 1.6870, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.335090
Epoch 110
Validation binary_cross_entropy = 0.294233
Epoch 111
Validation binary_cross_entropy = 0.305402
Epoch 112
Validation binary_cross_entropy = 0.400336
Epoch 113
Validation binary_cross_entropy = 0.464611
Epoch 114
Loss = 3.8443e-01, PNorm = 65.1662, GNorm = 1.5615, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.391361
Epoch 115
Validation binary_cross_entropy = 0.396203
Epoch 116
Validation binary_cross_entropy = 0.513497
Epoch 117
Validation binary_cross_entropy = 0.418178
Epoch 118
Validation binary_cross_entropy = 0.388487
Epoch 119
Loss = 2.5380e-01, PNorm = 65.2913, GNorm = 1.3731, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.613354
Epoch 120
Validation binary_cross_entropy = 0.474538
Epoch 121
Validation binary_cross_entropy = 0.372315
Epoch 122
Validation binary_cross_entropy = 0.433803
Epoch 123
Validation binary_cross_entropy = 0.484063
Epoch 124
Loss = 3.0119e-01, PNorm = 65.4197, GNorm = 1.3540, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.401010
Epoch 125
Validation binary_cross_entropy = 0.420512
Epoch 126
Validation binary_cross_entropy = 0.483132
Epoch 127
Validation binary_cross_entropy = 0.374444
Epoch 128
Validation binary_cross_entropy = 0.332217
Epoch 129
Loss = 2.8078e-01, PNorm = 65.5430, GNorm = 1.7484, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.350676
Epoch 130
Validation binary_cross_entropy = 0.404997
Epoch 131
Validation binary_cross_entropy = 0.444125
Epoch 132
Validation binary_cross_entropy = 0.494462
Epoch 133
Validation binary_cross_entropy = 0.512761
Epoch 134
Loss = 3.1399e-01, PNorm = 65.6717, GNorm = 3.2445, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.539115
Epoch 135
Validation binary_cross_entropy = 0.498464
Epoch 136
Validation binary_cross_entropy = 0.431225
Epoch 137
Validation binary_cross_entropy = 0.433785
Epoch 138
Validation binary_cross_entropy = 0.529801
Epoch 139
Loss = 2.8763e-01, PNorm = 65.7962, GNorm = 2.5216, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.535909
Epoch 140
Validation binary_cross_entropy = 0.414471
Epoch 141
Validation binary_cross_entropy = 0.440450
Epoch 142
Validation binary_cross_entropy = 0.442936
Epoch 143
Validation binary_cross_entropy = 0.478012
Epoch 144
Loss = 2.9299e-01, PNorm = 65.9132, GNorm = 1.2119, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.443291
Epoch 145
Validation binary_cross_entropy = 0.427315
Epoch 146
Validation binary_cross_entropy = 0.428895
Epoch 147
Validation binary_cross_entropy = 0.439332
Epoch 148
Validation binary_cross_entropy = 0.470202
Epoch 149
Loss = 1.4828e-01, PNorm = 66.0316, GNorm = 1.2197, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.520475
Epoch 150
Validation binary_cross_entropy = 0.561264
Epoch 151
Validation binary_cross_entropy = 0.590115
Epoch 152
Validation binary_cross_entropy = 0.623507
Epoch 153
Validation binary_cross_entropy = 0.638113
Epoch 154
Loss = 1.7463e-01, PNorm = 66.1310, GNorm = 1.1854, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.613183
Epoch 155
Validation binary_cross_entropy = 0.607485
Epoch 156
Validation binary_cross_entropy = 0.623779
Epoch 157
Validation binary_cross_entropy = 0.591585
Epoch 158
Validation binary_cross_entropy = 0.534203
Epoch 159
Loss = 2.1993e-01, PNorm = 66.2261, GNorm = 1.2403, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.540053
Epoch 160
Validation binary_cross_entropy = 0.528178
Epoch 161
Validation binary_cross_entropy = 0.534948
Epoch 162
Validation binary_cross_entropy = 0.557417
Epoch 163
Validation binary_cross_entropy = 0.638133
Epoch 164
Loss = 2.6333e-01, PNorm = 66.3243, GNorm = 4.8232, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.555892
Epoch 165
Validation binary_cross_entropy = 0.514741
Epoch 166
Validation binary_cross_entropy = 0.492567
Epoch 167
Validation binary_cross_entropy = 0.512404
Epoch 168
Validation binary_cross_entropy = 0.558871
Epoch 169
Loss = 1.7197e-01, PNorm = 66.4210, GNorm = 2.6101, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.562796
Epoch 170
Validation binary_cross_entropy = 0.608897
Epoch 171
Validation binary_cross_entropy = 0.645343
Epoch 172
Validation binary_cross_entropy = 0.671981
Epoch 173
Validation binary_cross_entropy = 0.648177
Epoch 174
Loss = 1.5521e-01, PNorm = 66.5116, GNorm = 0.7514, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 0.655749
Epoch 175
Validation binary_cross_entropy = 0.665171
Epoch 176
Validation binary_cross_entropy = 0.688922
Epoch 177
Validation binary_cross_entropy = 0.709575
Epoch 178
Validation binary_cross_entropy = 0.716942
Epoch 179
Loss = 1.1609e-01, PNorm = 66.6013, GNorm = 0.9371, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 0.663592
Epoch 180
Validation binary_cross_entropy = 0.680874
Epoch 181
Validation binary_cross_entropy = 0.772537
Epoch 182
Validation binary_cross_entropy = 0.786908
Epoch 183
Validation binary_cross_entropy = 0.815885
Epoch 184
Loss = 9.7833e-02, PNorm = 66.6867, GNorm = 2.0965, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 0.829068
Epoch 185
Validation binary_cross_entropy = 0.782669
Epoch 186
Validation binary_cross_entropy = 0.758480
Epoch 187
Validation binary_cross_entropy = 0.709322
Epoch 188
Validation binary_cross_entropy = 0.703032
Epoch 189
Loss = 1.7394e-01, PNorm = 66.7703, GNorm = 3.2816, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 0.664423
Epoch 190
Validation binary_cross_entropy = 0.684821
Epoch 191
Validation binary_cross_entropy = 0.642785
Epoch 192
Validation binary_cross_entropy = 0.630588
Epoch 193
Validation binary_cross_entropy = 0.667887
Epoch 194
Loss = 1.6007e-01, PNorm = 66.8688, GNorm = 1.3725, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 0.760205
Epoch 195
Validation binary_cross_entropy = 0.686413
Epoch 196
Validation binary_cross_entropy = 0.668900
Epoch 197
Validation binary_cross_entropy = 0.672013
Epoch 198
Validation binary_cross_entropy = 0.723598
Epoch 199
Loss = 1.6143e-01, PNorm = 66.9533, GNorm = 1.6249, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 0.691179
Epoch 200
Validation binary_cross_entropy = 0.662706
Epoch 201
Validation binary_cross_entropy = 0.672972
Epoch 202
Validation binary_cross_entropy = 0.682454
Epoch 203
Validation binary_cross_entropy = 0.722743
Epoch 204
Loss = 1.0428e-01, PNorm = 67.0344, GNorm = 1.3031, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 0.801398
Epoch 205
Validation binary_cross_entropy = 0.842544
Epoch 206
Validation binary_cross_entropy = 0.849514
Epoch 207
Validation binary_cross_entropy = 0.890061
Epoch 208
Validation binary_cross_entropy = 0.914719
Epoch 209
Loss = 1.1801e-01, PNorm = 67.0975, GNorm = 4.6481, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 0.888746
Epoch 210
Validation binary_cross_entropy = 0.834803
Epoch 211
Validation binary_cross_entropy = 0.835002
Epoch 212
Validation binary_cross_entropy = 0.895135
Epoch 213
Validation binary_cross_entropy = 0.831179
Epoch 214
Loss = 1.0330e-01, PNorm = 67.1638, GNorm = 3.2304, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 0.716137
Epoch 215
Validation binary_cross_entropy = 0.673215
Epoch 216
Validation binary_cross_entropy = 0.685730
Epoch 217
Validation binary_cross_entropy = 0.789129
Epoch 218
Validation binary_cross_entropy = 0.789408
Epoch 219
Loss = 6.2000e-02, PNorm = 67.2472, GNorm = 0.5601, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 0.766084
Epoch 220
Validation binary_cross_entropy = 0.781463
Epoch 221
Validation binary_cross_entropy = 0.805680
Epoch 222
Validation binary_cross_entropy = 0.860668
Epoch 223
Validation binary_cross_entropy = 0.866936
Epoch 224
Loss = 8.2610e-02, PNorm = 67.3192, GNorm = 0.6525, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 0.868158
Epoch 225
Validation binary_cross_entropy = 0.854680
Epoch 226
Validation binary_cross_entropy = 0.852938
Epoch 227
Validation binary_cross_entropy = 0.875112
Epoch 228
Validation binary_cross_entropy = 0.890244
Epoch 229
Loss = 8.9978e-02, PNorm = 67.3817, GNorm = 1.5746, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 0.952227
Epoch 230
Validation binary_cross_entropy = 1.043060
Epoch 231
Validation binary_cross_entropy = 0.976974
Epoch 232
Validation binary_cross_entropy = 0.938446
Epoch 233
Validation binary_cross_entropy = 0.943160
Epoch 234
Loss = 7.4074e-02, PNorm = 67.4410, GNorm = 0.7492, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 0.959421
Epoch 235
Validation binary_cross_entropy = 0.958774
Epoch 236
Validation binary_cross_entropy = 0.969639
Epoch 237
Validation binary_cross_entropy = 1.002702
Epoch 238
Validation binary_cross_entropy = 0.959959
Epoch 239
Loss = 8.3489e-02, PNorm = 67.4944, GNorm = 2.3913, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 0.937509
Epoch 240
Validation binary_cross_entropy = 1.004860
Epoch 241
Validation binary_cross_entropy = 1.049602
Epoch 242
Validation binary_cross_entropy = 0.968815
Epoch 243
Validation binary_cross_entropy = 0.923934
Epoch 244
Loss = 6.5318e-02, PNorm = 67.5451, GNorm = 0.8162, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 0.849236
Epoch 245
Validation binary_cross_entropy = 0.839018
Epoch 246
Validation binary_cross_entropy = 0.783737
Epoch 247
Validation binary_cross_entropy = 0.737618
Epoch 248
Validation binary_cross_entropy = 0.742191
Epoch 249
Loss = 1.4853e-01, PNorm = 67.6103, GNorm = 1.6332, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 0.802002
Epoch 250
Validation binary_cross_entropy = 0.864917
Epoch 251
Validation binary_cross_entropy = 0.907121
Epoch 252
Validation binary_cross_entropy = 0.939002
Epoch 253
Validation binary_cross_entropy = 0.971051
Epoch 254
Loss = 4.7866e-02, PNorm = 67.6775, GNorm = 1.4719, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 1.015588
Epoch 255
Validation binary_cross_entropy = 1.068408
Epoch 256
Validation binary_cross_entropy = 1.134671
Epoch 257
Validation binary_cross_entropy = 1.128936
Epoch 258
Validation binary_cross_entropy = 1.092734
Epoch 259
Loss = 1.7808e-01, PNorm = 67.7298, GNorm = 1.2032, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 1.060179
Epoch 260
Validation binary_cross_entropy = 1.047304
Epoch 261
Validation binary_cross_entropy = 1.022848
Epoch 262
Validation binary_cross_entropy = 0.923475
Epoch 263
Validation binary_cross_entropy = 0.931697
Epoch 264
Loss = 2.0036e-01, PNorm = 67.7981, GNorm = 2.0497, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 0.831905
Epoch 265
Validation binary_cross_entropy = 0.845620
Epoch 266
Validation binary_cross_entropy = 0.843334
Epoch 267
Validation binary_cross_entropy = 0.744547
Epoch 268
Validation binary_cross_entropy = 0.699488
Epoch 269
Loss = 1.1784e-01, PNorm = 67.8713, GNorm = 2.3271, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 0.713096
Epoch 270
Validation binary_cross_entropy = 0.745419
Epoch 271
Validation binary_cross_entropy = 0.795106
Epoch 272
Validation binary_cross_entropy = 0.849127
Epoch 273
Validation binary_cross_entropy = 0.905021
Epoch 274
Loss = 5.1612e-02, PNorm = 67.9433, GNorm = 0.9625, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 0.913278
Epoch 275
Validation binary_cross_entropy = 0.957402
Epoch 276
Validation binary_cross_entropy = 0.996785
Epoch 277
Validation binary_cross_entropy = 1.122777
Epoch 278
Validation binary_cross_entropy = 1.158667
Epoch 279
Loss = 7.0621e-02, PNorm = 68.0026, GNorm = 3.4434, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 1.043021
Epoch 280
Validation binary_cross_entropy = 0.984485
Epoch 281
Validation binary_cross_entropy = 0.949563
Epoch 282
Validation binary_cross_entropy = 0.933258
Epoch 283
Validation binary_cross_entropy = 0.970764
Epoch 284
Loss = 1.0843e-01, PNorm = 68.0577, GNorm = 3.3861, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 0.949165
Epoch 285
Validation binary_cross_entropy = 0.904148
Epoch 286
Validation binary_cross_entropy = 0.869145
Epoch 287
Validation binary_cross_entropy = 0.875050
Epoch 288
Validation binary_cross_entropy = 0.931237
Epoch 289
Loss = 5.5501e-02, PNorm = 68.1115, GNorm = 1.0132, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 0.946712
Epoch 290
Validation binary_cross_entropy = 0.925204
Epoch 291
Validation binary_cross_entropy = 0.921000
Epoch 292
Validation binary_cross_entropy = 0.942452
Epoch 293
Validation binary_cross_entropy = 1.019716
Epoch 294
Loss = 8.9854e-02, PNorm = 68.1630, GNorm = 2.4913, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 1.182991
Epoch 295
Validation binary_cross_entropy = 0.987588
Epoch 296
Validation binary_cross_entropy = 0.935637
Epoch 297
Validation binary_cross_entropy = 0.931514
Epoch 298
Validation binary_cross_entropy = 0.905639
Epoch 299
Loss = 1.6295e-01, PNorm = 68.2377, GNorm = 1.3616, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 0.962049
Model 0 best validation binary_cross_entropy = 0.291358 on epoch 7
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.242493
Ensemble test binary_cross_entropy = 0.242493
Fold 1
Splitting data with seed 1
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.4, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=800, bias=False)
        (W_h): Linear(in_features=800, out_features=800, bias=False)
        (W_o): Linear(in_features=933, out_features=800, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.4, inplace=False)
    (1): Linear(in_features=3100, out_features=800, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.4, inplace=False)
    (4): Linear(in_features=800, out_features=800, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.4, inplace=False)
    (7): Linear(in_features=800, out_features=1, bias=True)
  )
)
Number of parameters = 4,627,201
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 0.289308
Epoch 1
Validation binary_cross_entropy = 0.308339
Epoch 2
Validation binary_cross_entropy = 0.745188
Epoch 3
Validation binary_cross_entropy = 0.310702
Epoch 4
Loss = 9.0577e-01, PNorm = 63.1501, GNorm = 10.4012, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 0.297426
Epoch 5
Validation binary_cross_entropy = 0.490699
Epoch 6
Validation binary_cross_entropy = 0.968219
Epoch 7
Validation binary_cross_entropy = 0.325563
Epoch 8
Validation binary_cross_entropy = 0.300375
Epoch 9
Loss = 7.2111e-01, PNorm = 63.2884, GNorm = 3.8343, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 0.761792
Epoch 10
Validation binary_cross_entropy = 0.812666
Epoch 11
Validation binary_cross_entropy = 0.304944
Epoch 12
Validation binary_cross_entropy = 0.370894
Epoch 13
Validation binary_cross_entropy = 0.966551
Epoch 14
Loss = 8.0977e-01, PNorm = 63.4289, GNorm = 5.2333, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 0.645028
Epoch 15
Validation binary_cross_entropy = 0.397379
Epoch 16
Validation binary_cross_entropy = 0.381264
Epoch 17
Validation binary_cross_entropy = 0.597862
Epoch 18
Validation binary_cross_entropy = 0.714151
Epoch 19
Loss = 6.9391e-01, PNorm = 63.5343, GNorm = 2.6725, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 0.397722
Epoch 20
Validation binary_cross_entropy = 0.372812
Epoch 21
Validation binary_cross_entropy = 0.542879
Epoch 22
Validation binary_cross_entropy = 0.724101
Epoch 23
Validation binary_cross_entropy = 0.487529
Epoch 24
Loss = 6.7108e-01, PNorm = 63.6206, GNorm = 3.4559, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 0.374869
Epoch 25
Validation binary_cross_entropy = 0.444442
Epoch 26
Validation binary_cross_entropy = 0.608225
Epoch 27
Validation binary_cross_entropy = 0.614403
Epoch 28
Validation binary_cross_entropy = 0.490906
Epoch 29
Loss = 6.3944e-01, PNorm = 63.6893, GNorm = 3.5720, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 0.468085
Epoch 30
Validation binary_cross_entropy = 0.588779
Epoch 31
Validation binary_cross_entropy = 0.543842
Epoch 32
Validation binary_cross_entropy = 0.345004
Epoch 33
Validation binary_cross_entropy = 0.393434
Epoch 34
Loss = 5.5787e-01, PNorm = 63.7513, GNorm = 2.3707, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 0.595307
Epoch 35
Validation binary_cross_entropy = 0.634573
Epoch 36
Validation binary_cross_entropy = 0.531896
Epoch 37
Validation binary_cross_entropy = 0.447408
Epoch 38
Validation binary_cross_entropy = 0.468821
Epoch 39
Loss = 6.9781e-01, PNorm = 63.8134, GNorm = 2.5434, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 0.568841
Epoch 40
Validation binary_cross_entropy = 0.593225
Epoch 41
Validation binary_cross_entropy = 0.540140
Epoch 42
Validation binary_cross_entropy = 0.507117
Epoch 43
Validation binary_cross_entropy = 0.587334
Epoch 44
Loss = 5.6295e-01, PNorm = 63.8690, GNorm = 3.0899, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 0.534266
Epoch 45
Validation binary_cross_entropy = 0.423658
Epoch 46
Validation binary_cross_entropy = 0.440844
Epoch 47
Validation binary_cross_entropy = 0.539909
Epoch 48
Validation binary_cross_entropy = 0.626085
Epoch 49
Loss = 6.2883e-01, PNorm = 63.9222, GNorm = 2.6025, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 0.597083
Epoch 50
Validation binary_cross_entropy = 0.487702
Epoch 51
Validation binary_cross_entropy = 0.489121
Epoch 52
Validation binary_cross_entropy = 0.583457
Epoch 53
Validation binary_cross_entropy = 0.522685
Epoch 54
Loss = 6.3797e-01, PNorm = 63.9804, GNorm = 2.1524, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 0.445645
Epoch 55
Validation binary_cross_entropy = 0.417933
Epoch 56
Validation binary_cross_entropy = 0.494369
Epoch 57
Validation binary_cross_entropy = 0.561727
Epoch 58
Validation binary_cross_entropy = 0.504752
Epoch 59
Loss = 5.6181e-01, PNorm = 64.0434, GNorm = 1.8386, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 0.511117
Epoch 60
Validation binary_cross_entropy = 0.512990
Epoch 61
Validation binary_cross_entropy = 0.410916
Epoch 62
Validation binary_cross_entropy = 0.425751
Epoch 63
Validation binary_cross_entropy = 0.399608
Epoch 64
Loss = 5.9481e-01, PNorm = 64.1133, GNorm = 1.7818, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 0.447627
Epoch 65
Validation binary_cross_entropy = 0.585543
Epoch 66
Validation binary_cross_entropy = 0.498073
Epoch 67
Validation binary_cross_entropy = 0.397328
Epoch 68
Validation binary_cross_entropy = 0.532170
Epoch 69
Loss = 5.3980e-01, PNorm = 64.1840, GNorm = 2.5068, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 0.617519
Epoch 70
Validation binary_cross_entropy = 0.357128
Epoch 71
Validation binary_cross_entropy = 0.362392
Epoch 72
Validation binary_cross_entropy = 0.513583
Epoch 73
Validation binary_cross_entropy = 0.645172
Epoch 74
Loss = 5.5613e-01, PNorm = 64.2727, GNorm = 1.7584, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 0.434225
Epoch 75
Validation binary_cross_entropy = 0.372060
Epoch 76
Validation binary_cross_entropy = 0.436520
Epoch 77
Validation binary_cross_entropy = 0.467176
Epoch 78
Validation binary_cross_entropy = 0.487819
Epoch 79
Loss = 4.6261e-01, PNorm = 64.3592, GNorm = 1.7598, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 0.466109
Epoch 80
Validation binary_cross_entropy = 0.632748
Epoch 81
Validation binary_cross_entropy = 0.433313
Epoch 82
Validation binary_cross_entropy = 0.361331
Epoch 83
Validation binary_cross_entropy = 0.537557
Epoch 84
Loss = 4.9723e-01, PNorm = 64.4520, GNorm = 2.1733, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 0.431467
Epoch 85
Validation binary_cross_entropy = 0.333543
Epoch 86
Validation binary_cross_entropy = 0.370174
Epoch 87
Validation binary_cross_entropy = 0.555276
Epoch 88
Validation binary_cross_entropy = 0.479054
Epoch 89
Loss = 4.5249e-01, PNorm = 64.5630, GNorm = 1.4847, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 0.329159
Epoch 90
Validation binary_cross_entropy = 0.380013
Epoch 91
Validation binary_cross_entropy = 0.409475
Epoch 92
Validation binary_cross_entropy = 0.364087
Epoch 93
Validation binary_cross_entropy = 0.476619
Epoch 94
Loss = 4.5156e-01, PNorm = 64.6781, GNorm = 1.3966, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 0.400531
Epoch 95
Validation binary_cross_entropy = 0.328011
Epoch 96
Validation binary_cross_entropy = 0.397281
Epoch 97
Validation binary_cross_entropy = 0.442510
Epoch 98
Validation binary_cross_entropy = 0.316867
Epoch 99
Loss = 3.9917e-01, PNorm = 64.7948, GNorm = 1.3384, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.381642
Epoch 100
Validation binary_cross_entropy = 0.340656
Epoch 101
Validation binary_cross_entropy = 0.343807
Epoch 102
Validation binary_cross_entropy = 0.409868
Epoch 103
Validation binary_cross_entropy = 0.331273
Epoch 104
Loss = 4.1234e-01, PNorm = 64.9178, GNorm = 2.1351, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.335633
Epoch 105
Validation binary_cross_entropy = 0.332484
Epoch 106
Validation binary_cross_entropy = 0.463478
Epoch 107
Validation binary_cross_entropy = 0.348700
Epoch 108
Validation binary_cross_entropy = 0.437580
Epoch 109
Loss = 3.7253e-01, PNorm = 65.0462, GNorm = 2.3925, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.392550
Epoch 110
Validation binary_cross_entropy = 0.364847
Epoch 111
Validation binary_cross_entropy = 0.437888
Epoch 112
Validation binary_cross_entropy = 0.386473
Epoch 113
Validation binary_cross_entropy = 0.712884
Epoch 114
Loss = 5.3870e-01, PNorm = 65.1742, GNorm = 4.7194, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.529560
Epoch 115
Validation binary_cross_entropy = 0.287942
Epoch 116
Validation binary_cross_entropy = 0.284855
Epoch 117
Validation binary_cross_entropy = 0.348772
Epoch 118
Validation binary_cross_entropy = 0.457036
Epoch 119
Loss = 4.3748e-01, PNorm = 65.3078, GNorm = 2.5577, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.349689
Epoch 120
Validation binary_cross_entropy = 0.314681
Epoch 121
Validation binary_cross_entropy = 0.337326
Epoch 122
Validation binary_cross_entropy = 0.412077
Epoch 123
Validation binary_cross_entropy = 0.388731
Epoch 124
Loss = 2.5967e-01, PNorm = 65.4271, GNorm = 1.8596, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.376822
Epoch 125
Validation binary_cross_entropy = 0.444676
Epoch 126
Validation binary_cross_entropy = 0.428776
Epoch 127
Validation binary_cross_entropy = 0.376223
Epoch 128
Validation binary_cross_entropy = 0.440184
Epoch 129
Loss = 3.0453e-01, PNorm = 65.5484, GNorm = 2.1337, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.404032
Epoch 130
Validation binary_cross_entropy = 0.382363
Epoch 131
Validation binary_cross_entropy = 0.387257
Epoch 132
Validation binary_cross_entropy = 0.453730
Epoch 133
Validation binary_cross_entropy = 0.388469
Epoch 134
Loss = 2.5706e-01, PNorm = 65.6563, GNorm = 3.7271, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.397460
Epoch 135
Validation binary_cross_entropy = 0.572592
Epoch 136
Validation binary_cross_entropy = 0.415203
Epoch 137
Validation binary_cross_entropy = 0.468749
Epoch 138
Validation binary_cross_entropy = 0.418509
Epoch 139
Loss = 2.2561e-01, PNorm = 65.7830, GNorm = 2.0765, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.517096
Epoch 140
Validation binary_cross_entropy = 0.434145
Epoch 141
Validation binary_cross_entropy = 0.436579
Epoch 142
Validation binary_cross_entropy = 0.438330
Epoch 143
Validation binary_cross_entropy = 0.451210
Epoch 144
Loss = 3.0741e-01, PNorm = 65.9027, GNorm = 1.8342, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.466161
Epoch 145
Validation binary_cross_entropy = 0.447815
Epoch 146
Validation binary_cross_entropy = 0.477796
Epoch 147
Validation binary_cross_entropy = 0.529831
Epoch 148
Validation binary_cross_entropy = 0.580780
Epoch 149
Loss = 2.0592e-01, PNorm = 66.0056, GNorm = 2.2377, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.587712
Epoch 150
Validation binary_cross_entropy = 0.592715
Epoch 151
Validation binary_cross_entropy = 0.583533
Epoch 152
Validation binary_cross_entropy = 0.536556
Epoch 153
Validation binary_cross_entropy = 0.515101
Epoch 154
Loss = 1.4351e-01, PNorm = 66.1022, GNorm = 1.7629, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.518773
Epoch 155
Validation binary_cross_entropy = 0.568467
Epoch 156
Validation binary_cross_entropy = 0.541344
Epoch 157
Validation binary_cross_entropy = 0.601563
Epoch 158
Validation binary_cross_entropy = 0.555374
Epoch 159
Loss = 2.1450e-01, PNorm = 66.2043, GNorm = 4.5361, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.572070
Epoch 160
Validation binary_cross_entropy = 0.543611
Epoch 161
Validation binary_cross_entropy = 0.550204
Epoch 162
Validation binary_cross_entropy = 0.446803
Epoch 163
Validation binary_cross_entropy = 0.501269
Epoch 164
Loss = 2.9160e-01, PNorm = 66.3242, GNorm = 4.4682, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.499625
Epoch 165
Validation binary_cross_entropy = 0.525995
Epoch 166
Validation binary_cross_entropy = 0.536545
Epoch 167
Validation binary_cross_entropy = 0.516530
Epoch 168
Validation binary_cross_entropy = 0.589170
Epoch 169
Loss = 2.2057e-01, PNorm = 66.4346, GNorm = 4.6080, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.590660
Epoch 170
Validation binary_cross_entropy = 0.562795
Epoch 171
Validation binary_cross_entropy = 0.546553
Epoch 172
Validation binary_cross_entropy = 0.573984
Epoch 173
Validation binary_cross_entropy = 0.605069
Epoch 174
Loss = 1.7019e-01, PNorm = 66.5300, GNorm = 1.2926, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 0.552804
Epoch 175
Validation binary_cross_entropy = 0.570557
Epoch 176
Validation binary_cross_entropy = 0.635902
Epoch 177
Validation binary_cross_entropy = 0.724224
Epoch 178
Validation binary_cross_entropy = 0.633745
Epoch 179
Loss = 2.8175e-01, PNorm = 66.6204, GNorm = 5.6135, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 0.553052
Epoch 180
Validation binary_cross_entropy = 0.524558
Epoch 181
Validation binary_cross_entropy = 0.545257
Epoch 182
Validation binary_cross_entropy = 0.465571
Epoch 183
Validation binary_cross_entropy = 0.468171
Epoch 184
Loss = 2.1440e-01, PNorm = 66.7146, GNorm = 1.2290, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 0.469993
Epoch 185
Validation binary_cross_entropy = 0.505197
Epoch 186
Validation binary_cross_entropy = 0.568072
Epoch 187
Validation binary_cross_entropy = 0.629627
Epoch 188
Validation binary_cross_entropy = 0.684553
Epoch 189
Loss = 2.8195e-01, PNorm = 66.8107, GNorm = 2.7529, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 0.707812
Epoch 190
Validation binary_cross_entropy = 0.688941
Epoch 191
Validation binary_cross_entropy = 0.684806
Epoch 192
Validation binary_cross_entropy = 0.663233
Epoch 193
Validation binary_cross_entropy = 0.645649
Epoch 194
Loss = 1.9146e-01, PNorm = 66.8932, GNorm = 3.0490, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 0.657359
Epoch 195
Validation binary_cross_entropy = 0.753763
Epoch 196
Validation binary_cross_entropy = 0.693986
Epoch 197
Validation binary_cross_entropy = 0.653661
Epoch 198
Validation binary_cross_entropy = 0.668118
Epoch 199
Loss = 1.1276e-01, PNorm = 66.9779, GNorm = 0.8511, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 0.694273
Epoch 200
Validation binary_cross_entropy = 0.756639
Epoch 201
Validation binary_cross_entropy = 0.799316
Epoch 202
Validation binary_cross_entropy = 0.719957
Epoch 203
Validation binary_cross_entropy = 0.793104
Epoch 204
Loss = 3.1371e-01, PNorm = 67.0476, GNorm = 3.4155, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 0.636993
Epoch 205
Validation binary_cross_entropy = 0.687180
Epoch 206
Validation binary_cross_entropy = 0.666155
Epoch 207
Validation binary_cross_entropy = 0.614224
Epoch 208
Validation binary_cross_entropy = 0.625573
Epoch 209
Loss = 1.3771e-01, PNorm = 67.1311, GNorm = 1.6311, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 0.673266
Epoch 210
Validation binary_cross_entropy = 0.760616
Epoch 211
Validation binary_cross_entropy = 0.844658
Epoch 212
Validation binary_cross_entropy = 0.758349
Epoch 213
Validation binary_cross_entropy = 0.674207
Epoch 214
Loss = 4.7765e-02, PNorm = 67.2067, GNorm = 0.3662, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 0.639954
Epoch 215
Validation binary_cross_entropy = 0.654130
Epoch 216
Validation binary_cross_entropy = 0.721411
Epoch 217
Validation binary_cross_entropy = 0.782650
Epoch 218
Validation binary_cross_entropy = 0.826679
Epoch 219
Loss = 1.8839e-01, PNorm = 67.2747, GNorm = 1.4270, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 0.826580
Epoch 220
Validation binary_cross_entropy = 0.857735
Epoch 221
Validation binary_cross_entropy = 0.909648
Epoch 222
Validation binary_cross_entropy = 0.921396
Epoch 223
Validation binary_cross_entropy = 0.859659
Epoch 224
Loss = 2.3139e-01, PNorm = 67.3357, GNorm = 3.3838, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 0.872573
Epoch 225
Validation binary_cross_entropy = 0.944735
Epoch 226
Validation binary_cross_entropy = 0.789036
Epoch 227
Validation binary_cross_entropy = 0.778644
Epoch 228
Validation binary_cross_entropy = 0.779214
Epoch 229
Loss = 6.5032e-02, PNorm = 67.3997, GNorm = 0.7788, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 0.818322
Epoch 230
Validation binary_cross_entropy = 0.835143
Epoch 231
Validation binary_cross_entropy = 0.828347
Epoch 232
Validation binary_cross_entropy = 0.844112
Epoch 233
Validation binary_cross_entropy = 0.858756
Epoch 234
Loss = 6.4748e-02, PNorm = 67.4588, GNorm = 1.2652, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 0.882232
Epoch 235
Validation binary_cross_entropy = 0.945686
Epoch 236
Validation binary_cross_entropy = 1.016312
Epoch 237
Validation binary_cross_entropy = 1.039401
Epoch 238
Validation binary_cross_entropy = 1.013759
Epoch 239
Loss = 7.7572e-02, PNorm = 67.5192, GNorm = 2.9065, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 1.006788
Epoch 240
Validation binary_cross_entropy = 1.060846
Epoch 241
Validation binary_cross_entropy = 1.133138
Epoch 242
Validation binary_cross_entropy = 0.855320
Epoch 243
Validation binary_cross_entropy = 0.743544
Epoch 244
Loss = 2.1591e-01, PNorm = 67.5761, GNorm = 2.7385, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 0.687889
Epoch 245
Validation binary_cross_entropy = 0.700435
Epoch 246
Validation binary_cross_entropy = 0.726985
Epoch 247
Validation binary_cross_entropy = 0.696845
Epoch 248
Validation binary_cross_entropy = 0.688697
Epoch 249
Loss = 1.3927e-01, PNorm = 67.6470, GNorm = 1.8450, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 0.716639
Epoch 250
Validation binary_cross_entropy = 0.759529
Epoch 251
Validation binary_cross_entropy = 0.839970
Epoch 252
Validation binary_cross_entropy = 0.912774
Epoch 253
Validation binary_cross_entropy = 0.968920
Epoch 254
Loss = 5.6834e-02, PNorm = 67.7127, GNorm = 1.4193, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 1.024158
Epoch 255
Validation binary_cross_entropy = 1.100507
Epoch 256
Validation binary_cross_entropy = 1.124451
Epoch 257
Validation binary_cross_entropy = 1.151587
Epoch 258
Validation binary_cross_entropy = 1.168363
Epoch 259
Loss = 2.1847e-02, PNorm = 67.7691, GNorm = 0.3959, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 1.170577
Epoch 260
Validation binary_cross_entropy = 1.150499
Epoch 261
Validation binary_cross_entropy = 1.137046
Epoch 262
Validation binary_cross_entropy = 1.142108
Epoch 263
Validation binary_cross_entropy = 1.081942
Epoch 264
Loss = 3.4890e-02, PNorm = 67.8215, GNorm = 1.0332, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 1.185257
Epoch 265
Validation binary_cross_entropy = 1.208048
Epoch 266
Validation binary_cross_entropy = 1.112380
Epoch 267
Validation binary_cross_entropy = 1.048141
Epoch 268
Validation binary_cross_entropy = 0.969296
Epoch 269
Loss = 4.3590e-02, PNorm = 67.8811, GNorm = 2.0567, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 0.966447
Epoch 270
Validation binary_cross_entropy = 0.920465
Epoch 271
Validation binary_cross_entropy = 0.913264
Epoch 272
Validation binary_cross_entropy = 1.067521
Epoch 273
Validation binary_cross_entropy = 1.069868
Epoch 274
Loss = 2.7767e-02, PNorm = 67.9443, GNorm = 0.3622, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 0.933185
Epoch 275
Validation binary_cross_entropy = 0.899960
Epoch 276
Validation binary_cross_entropy = 0.886046
Epoch 277
Validation binary_cross_entropy = 0.918143
Epoch 278
Validation binary_cross_entropy = 0.954700
Epoch 279
Loss = 8.0904e-02, PNorm = 68.0034, GNorm = 1.4508, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 0.899228
Epoch 280
Validation binary_cross_entropy = 0.874867
Epoch 281
Validation binary_cross_entropy = 0.897311
Epoch 282
Validation binary_cross_entropy = 0.940683
Epoch 283
Validation binary_cross_entropy = 1.050726
Epoch 284
Loss = 5.3572e-02, PNorm = 68.0619, GNorm = 1.6224, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 1.137102
Epoch 285
Validation binary_cross_entropy = 1.132738
Epoch 286
Validation binary_cross_entropy = 1.153462
Epoch 287
Validation binary_cross_entropy = 1.176990
Epoch 288
Validation binary_cross_entropy = 1.233647
Epoch 289
Loss = 2.9371e-02, PNorm = 68.1240, GNorm = 0.7905, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 1.296764
Epoch 290
Validation binary_cross_entropy = 1.281099
Epoch 291
Validation binary_cross_entropy = 1.198955
Epoch 292
Validation binary_cross_entropy = 1.186077
Epoch 293
Validation binary_cross_entropy = 1.252288
Epoch 294
Loss = 3.9203e-02, PNorm = 68.1847, GNorm = 1.4202, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 1.259666
Epoch 295
Validation binary_cross_entropy = 1.139686
Epoch 296
Validation binary_cross_entropy = 1.104261
Epoch 297
Validation binary_cross_entropy = 1.093130
Epoch 298
Validation binary_cross_entropy = 1.075757
Epoch 299
Loss = 4.9867e-02, PNorm = 68.2485, GNorm = 0.8541, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 1.134324
Model 0 best validation binary_cross_entropy = 0.284855 on epoch 116
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.198337
Ensemble test binary_cross_entropy = 0.198337
Fold 2
Splitting data with seed 2
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.4, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=800, bias=False)
        (W_h): Linear(in_features=800, out_features=800, bias=False)
        (W_o): Linear(in_features=933, out_features=800, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.4, inplace=False)
    (1): Linear(in_features=3100, out_features=800, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.4, inplace=False)
    (4): Linear(in_features=800, out_features=800, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.4, inplace=False)
    (7): Linear(in_features=800, out_features=1, bias=True)
  )
)
Number of parameters = 4,627,201
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 0.289617
Epoch 1
Validation binary_cross_entropy = 0.304917
Epoch 2
Validation binary_cross_entropy = 0.601288
Epoch 3
Validation binary_cross_entropy = 0.420122
Epoch 4
Loss = 7.6577e-01, PNorm = 63.1502, GNorm = 7.0863, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 0.297528
Epoch 5
Validation binary_cross_entropy = 0.422881
Epoch 6
Validation binary_cross_entropy = 0.756407
Epoch 7
Validation binary_cross_entropy = 0.303197
Epoch 8
Validation binary_cross_entropy = 0.332482
Epoch 9
Loss = 8.5060e-01, PNorm = 63.2876, GNorm = 5.5403, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 0.913284
Epoch 10
Validation binary_cross_entropy = 0.310293
Epoch 11
Validation binary_cross_entropy = 0.325268
Epoch 12
Validation binary_cross_entropy = 0.718814
Epoch 13
Validation binary_cross_entropy = 0.341210
Epoch 14
Loss = 8.1144e-01, PNorm = 63.4388, GNorm = 5.7765, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 0.346163
Epoch 15
Validation binary_cross_entropy = 0.693508
Epoch 16
Validation binary_cross_entropy = 0.361234
Epoch 17
Validation binary_cross_entropy = 0.303745
Epoch 18
Validation binary_cross_entropy = 0.468845
Epoch 19
Loss = 6.2844e-01, PNorm = 63.5597, GNorm = 5.5798, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 0.659127
Epoch 20
Validation binary_cross_entropy = 0.324386
Epoch 21
Validation binary_cross_entropy = 0.304827
Epoch 22
Validation binary_cross_entropy = 0.408427
Epoch 23
Validation binary_cross_entropy = 0.990125
Epoch 24
Loss = 7.3082e-01, PNorm = 63.6433, GNorm = 5.6986, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 0.674218
Epoch 25
Validation binary_cross_entropy = 0.367297
Epoch 26
Validation binary_cross_entropy = 0.418726
Epoch 27
Validation binary_cross_entropy = 0.720029
Epoch 28
Validation binary_cross_entropy = 0.713908
Epoch 29
Loss = 5.8398e-01, PNorm = 63.7123, GNorm = 2.9650, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 0.530297
Epoch 30
Validation binary_cross_entropy = 0.520336
Epoch 31
Validation binary_cross_entropy = 0.568638
Epoch 32
Validation binary_cross_entropy = 0.549944
Epoch 33
Validation binary_cross_entropy = 0.533363
Epoch 34
Loss = 6.9430e-01, PNorm = 63.7777, GNorm = 2.6115, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 0.541354
Epoch 35
Validation binary_cross_entropy = 0.560324
Epoch 36
Validation binary_cross_entropy = 0.444775
Epoch 37
Validation binary_cross_entropy = 0.415333
Epoch 38
Validation binary_cross_entropy = 0.591142
Epoch 39
Loss = 6.3802e-01, PNorm = 63.8303, GNorm = 3.3378, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 0.675578
Epoch 40
Validation binary_cross_entropy = 0.511834
Epoch 41
Validation binary_cross_entropy = 0.443027
Epoch 42
Validation binary_cross_entropy = 0.516000
Epoch 43
Validation binary_cross_entropy = 0.583309
Epoch 44
Loss = 6.6222e-01, PNorm = 63.8797, GNorm = 3.1191, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 0.601022
Epoch 45
Validation binary_cross_entropy = 0.431168
Epoch 46
Validation binary_cross_entropy = 0.402526
Epoch 47
Validation binary_cross_entropy = 0.422764
Epoch 48
Validation binary_cross_entropy = 0.484612
Epoch 49
Loss = 6.1539e-01, PNorm = 63.9383, GNorm = 2.3557, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 0.517515
Epoch 50
Validation binary_cross_entropy = 0.467650
Epoch 51
Validation binary_cross_entropy = 0.433795
Epoch 52
Validation binary_cross_entropy = 0.488805
Epoch 53
Validation binary_cross_entropy = 0.514093
Epoch 54
Loss = 5.8996e-01, PNorm = 63.9949, GNorm = 2.0851, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 0.531979
Epoch 55
Validation binary_cross_entropy = 0.505982
Epoch 56
Validation binary_cross_entropy = 0.417945
Epoch 57
Validation binary_cross_entropy = 0.415940
Epoch 58
Validation binary_cross_entropy = 0.375466
Epoch 59
Loss = 6.2153e-01, PNorm = 64.0547, GNorm = 1.6428, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 0.461701
Epoch 60
Validation binary_cross_entropy = 0.625850
Epoch 61
Validation binary_cross_entropy = 0.607951
Epoch 62
Validation binary_cross_entropy = 0.487565
Epoch 63
Validation binary_cross_entropy = 0.463475
Epoch 64
Loss = 5.3557e-01, PNorm = 64.1314, GNorm = 2.1343, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 0.475087
Epoch 65
Validation binary_cross_entropy = 0.396542
Epoch 66
Validation binary_cross_entropy = 0.415175
Epoch 67
Validation binary_cross_entropy = 0.509714
Epoch 68
Validation binary_cross_entropy = 0.438522
Epoch 69
Loss = 5.2263e-01, PNorm = 64.2227, GNorm = 1.5761, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 0.438391
Epoch 70
Validation binary_cross_entropy = 0.499057
Epoch 71
Validation binary_cross_entropy = 0.364971
Epoch 72
Validation binary_cross_entropy = 0.558006
Epoch 73
Validation binary_cross_entropy = 0.470181
Epoch 74
Loss = 5.0133e-01, PNorm = 64.3167, GNorm = 1.3991, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 0.387823
Epoch 75
Validation binary_cross_entropy = 0.368294
Epoch 76
Validation binary_cross_entropy = 0.414867
Epoch 77
Validation binary_cross_entropy = 0.493050
Epoch 78
Validation binary_cross_entropy = 0.328329
Epoch 79
Loss = 5.2386e-01, PNorm = 64.4258, GNorm = 1.7192, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 0.390897
Epoch 80
Validation binary_cross_entropy = 0.488785
Epoch 81
Validation binary_cross_entropy = 0.270032
Epoch 82
Validation binary_cross_entropy = 0.436148
Epoch 83
Validation binary_cross_entropy = 0.542536
Epoch 84
Loss = 5.1344e-01, PNorm = 64.5311, GNorm = 1.3049, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 0.401678
Epoch 85
Validation binary_cross_entropy = 0.380955
Epoch 86
Validation binary_cross_entropy = 0.418707
Epoch 87
Validation binary_cross_entropy = 0.469891
Epoch 88
Validation binary_cross_entropy = 0.428597
Epoch 89
Loss = 5.1390e-01, PNorm = 64.6448, GNorm = 1.6964, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 0.346294
Epoch 90
Validation binary_cross_entropy = 0.355845
Epoch 91
Validation binary_cross_entropy = 0.488675
Epoch 92
Validation binary_cross_entropy = 0.337905
Epoch 93
Validation binary_cross_entropy = 0.352543
Epoch 94
Loss = 3.7831e-01, PNorm = 64.7592, GNorm = 1.8561, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 0.339665
Epoch 95
Validation binary_cross_entropy = 0.359025
Epoch 96
Validation binary_cross_entropy = 0.303552
Epoch 97
Validation binary_cross_entropy = 0.354828
Epoch 98
Validation binary_cross_entropy = 0.390987
Epoch 99
Loss = 3.7516e-01, PNorm = 64.8644, GNorm = 2.0720, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.314549
Epoch 100
Validation binary_cross_entropy = 0.422985
Epoch 101
Validation binary_cross_entropy = 0.389920
Epoch 102
Validation binary_cross_entropy = 0.365315
Epoch 103
Validation binary_cross_entropy = 0.429874
Epoch 104
Loss = 4.3461e-01, PNorm = 64.9902, GNorm = 1.4622, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.309185
Epoch 105
Validation binary_cross_entropy = 0.398383
Epoch 106
Validation binary_cross_entropy = 0.423090
Epoch 107
Validation binary_cross_entropy = 0.304349
Epoch 108
Validation binary_cross_entropy = 0.327756
Epoch 109
Loss = 3.9548e-01, PNorm = 65.1263, GNorm = 2.6429, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.373369
Epoch 110
Validation binary_cross_entropy = 0.360013
Epoch 111
Validation binary_cross_entropy = 0.585442
Epoch 112
Validation binary_cross_entropy = 0.465499
Epoch 113
Validation binary_cross_entropy = 0.406105
Epoch 114
Loss = 2.9206e-01, PNorm = 65.2615, GNorm = 1.6956, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.341595
Epoch 115
Validation binary_cross_entropy = 0.342887
Epoch 116
Validation binary_cross_entropy = 0.409773
Epoch 117
Validation binary_cross_entropy = 0.348576
Epoch 118
Validation binary_cross_entropy = 0.447500
Epoch 119
Loss = 3.0793e-01, PNorm = 65.4015, GNorm = 2.2321, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.439217
Epoch 120
Validation binary_cross_entropy = 0.368593
Epoch 121
Validation binary_cross_entropy = 0.426201
Epoch 122
Validation binary_cross_entropy = 0.394298
Epoch 123
Validation binary_cross_entropy = 0.365323
Epoch 124
Loss = 1.8249e-01, PNorm = 65.5519, GNorm = 1.0571, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.482644
Epoch 125
Validation binary_cross_entropy = 0.548106
Epoch 126
Validation binary_cross_entropy = 0.406232
Epoch 127
Validation binary_cross_entropy = 0.504324
Epoch 128
Validation binary_cross_entropy = 0.385373
Epoch 129
Loss = 3.2431e-01, PNorm = 65.6953, GNorm = 4.5602, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.474217
Epoch 130
Validation binary_cross_entropy = 0.518473
Epoch 131
Validation binary_cross_entropy = 0.352604
Epoch 132
Validation binary_cross_entropy = 0.368539
Epoch 133
Validation binary_cross_entropy = 0.583950
Epoch 134
Loss = 2.4005e-01, PNorm = 65.8387, GNorm = 1.7376, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.476216
Epoch 135
Validation binary_cross_entropy = 0.413791
Epoch 136
Validation binary_cross_entropy = 0.412420
Epoch 137
Validation binary_cross_entropy = 0.433675
Epoch 138
Validation binary_cross_entropy = 0.467763
Epoch 139
Loss = 3.2667e-01, PNorm = 65.9711, GNorm = 1.2877, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.376226
Epoch 140
Validation binary_cross_entropy = 0.396020
Epoch 141
Validation binary_cross_entropy = 0.548683
Epoch 142
Validation binary_cross_entropy = 0.477305
Epoch 143
Validation binary_cross_entropy = 0.432081
Epoch 144
Loss = 1.8912e-01, PNorm = 66.1027, GNorm = 2.6414, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.447471
Epoch 145
Validation binary_cross_entropy = 0.506178
Epoch 146
Validation binary_cross_entropy = 0.505655
Epoch 147
Validation binary_cross_entropy = 0.499793
Epoch 148
Validation binary_cross_entropy = 0.501598
Epoch 149
Loss = 1.8092e-01, PNorm = 66.2169, GNorm = 1.1772, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.529852
Epoch 150
Validation binary_cross_entropy = 0.546075
Epoch 151
Validation binary_cross_entropy = 0.540012
Epoch 152
Validation binary_cross_entropy = 0.537561
Epoch 153
Validation binary_cross_entropy = 0.575755
Epoch 154
Loss = 2.2818e-01, PNorm = 66.3099, GNorm = 1.2615, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.528691
Epoch 155
Validation binary_cross_entropy = 0.535657
Epoch 156
Validation binary_cross_entropy = 0.569985
Epoch 157
Validation binary_cross_entropy = 0.612173
Epoch 158
Validation binary_cross_entropy = 0.560559
Epoch 159
Loss = 2.5113e-01, PNorm = 66.4037, GNorm = 5.9152, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.550427
Epoch 160
Validation binary_cross_entropy = 0.531274
Epoch 161
Validation binary_cross_entropy = 0.502541
Epoch 162
Validation binary_cross_entropy = 0.477752
Epoch 163
Validation binary_cross_entropy = 0.479849
Epoch 164
Loss = 1.7858e-01, PNorm = 66.5064, GNorm = 2.3603, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.476712
Epoch 165
Validation binary_cross_entropy = 0.492389
Epoch 166
Validation binary_cross_entropy = 0.511867
Epoch 167
Validation binary_cross_entropy = 0.559460
Epoch 168
Validation binary_cross_entropy = 0.568172
Epoch 169
Loss = 2.3865e-01, PNorm = 66.6028, GNorm = 2.5571, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.573370
Epoch 170
Validation binary_cross_entropy = 0.582988
Epoch 171
Validation binary_cross_entropy = 0.611908
Epoch 172
Validation binary_cross_entropy = 0.662162
Epoch 173
Validation binary_cross_entropy = 0.648682
Epoch 174
Loss = 1.1630e-01, PNorm = 66.6882, GNorm = 2.2408, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 0.660404
Epoch 175
Validation binary_cross_entropy = 0.663350
Epoch 176
Validation binary_cross_entropy = 0.688907
Epoch 177
Validation binary_cross_entropy = 0.699619
Epoch 178
Validation binary_cross_entropy = 0.737166
Epoch 179
Loss = 1.7644e-01, PNorm = 66.7722, GNorm = 1.9250, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 0.753317
Epoch 180
Validation binary_cross_entropy = 0.752408
Epoch 181
Validation binary_cross_entropy = 0.727404
Epoch 182
Validation binary_cross_entropy = 0.722525
Epoch 183
Validation binary_cross_entropy = 0.697629
Epoch 184
Loss = 1.6365e-01, PNorm = 66.8553, GNorm = 0.9590, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 0.704325
Epoch 185
Validation binary_cross_entropy = 0.684196
Epoch 186
Validation binary_cross_entropy = 0.661657
Epoch 187
Validation binary_cross_entropy = 0.680440
Epoch 188
Validation binary_cross_entropy = 0.660377
Epoch 189
Loss = 1.4425e-01, PNorm = 66.9331, GNorm = 2.3267, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 0.660017
Epoch 190
Validation binary_cross_entropy = 0.669937
Epoch 191
Validation binary_cross_entropy = 0.702215
Epoch 192
Validation binary_cross_entropy = 0.741260
Epoch 193
Validation binary_cross_entropy = 0.767917
Epoch 194
Loss = 9.1339e-02, PNorm = 67.0233, GNorm = 2.3180, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 0.801477
Epoch 195
Validation binary_cross_entropy = 0.857411
Epoch 196
Validation binary_cross_entropy = 0.707694
Epoch 197
Validation binary_cross_entropy = 0.659324
Epoch 198
Validation binary_cross_entropy = 0.619669
Epoch 199
Loss = 1.4107e-01, PNorm = 67.1039, GNorm = 1.0598, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 0.607717
Epoch 200
Validation binary_cross_entropy = 0.597727
Epoch 201
Validation binary_cross_entropy = 0.631146
Epoch 202
Validation binary_cross_entropy = 0.660322
Epoch 203
Validation binary_cross_entropy = 0.783280
Epoch 204
Loss = 1.8524e-01, PNorm = 67.1870, GNorm = 4.2221, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 0.797089
Epoch 205
Validation binary_cross_entropy = 0.795312
Epoch 206
Validation binary_cross_entropy = 0.851332
Epoch 207
Validation binary_cross_entropy = 0.918376
Epoch 208
Validation binary_cross_entropy = 0.792847
Epoch 209
Loss = 4.6530e-02, PNorm = 67.2745, GNorm = 0.6507, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 0.751124
Epoch 210
Validation binary_cross_entropy = 0.682671
Epoch 211
Validation binary_cross_entropy = 0.642507
Epoch 212
Validation binary_cross_entropy = 0.636322
Epoch 213
Validation binary_cross_entropy = 0.634993
Epoch 214
Loss = 1.8258e-01, PNorm = 67.3645, GNorm = 1.3987, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 0.644929
Epoch 215
Validation binary_cross_entropy = 0.612446
Epoch 216
Validation binary_cross_entropy = 0.651779
Epoch 217
Validation binary_cross_entropy = 0.687839
Epoch 218
Validation binary_cross_entropy = 0.687274
Epoch 219
Loss = 1.3855e-01, PNorm = 67.4513, GNorm = 1.8090, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 0.788704
Epoch 220
Validation binary_cross_entropy = 0.675384
Epoch 221
Validation binary_cross_entropy = 0.712676
Epoch 222
Validation binary_cross_entropy = 0.668932
Epoch 223
Validation binary_cross_entropy = 0.587459
Epoch 224
Loss = 9.2291e-02, PNorm = 67.5186, GNorm = 0.8868, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 0.575271
Epoch 225
Validation binary_cross_entropy = 0.580232
Epoch 226
Validation binary_cross_entropy = 0.606595
Epoch 227
Validation binary_cross_entropy = 0.668242
Epoch 228
Validation binary_cross_entropy = 0.694999
Epoch 229
Loss = 8.1139e-02, PNorm = 67.5856, GNorm = 1.0513, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 0.729251
Epoch 230
Validation binary_cross_entropy = 0.772452
Epoch 231
Validation binary_cross_entropy = 0.809136
Epoch 232
Validation binary_cross_entropy = 0.830582
Epoch 233
Validation binary_cross_entropy = 0.828586
Epoch 234
Loss = 5.7201e-02, PNorm = 67.6479, GNorm = 2.1207, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 0.850209
Epoch 235
Validation binary_cross_entropy = 0.885590
Epoch 236
Validation binary_cross_entropy = 0.929206
Epoch 237
Validation binary_cross_entropy = 0.936283
Epoch 238
Validation binary_cross_entropy = 0.953206
Epoch 239
Loss = 1.3627e-01, PNorm = 67.7015, GNorm = 4.9084, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 0.946843
Epoch 240
Validation binary_cross_entropy = 0.904491
Epoch 241
Validation binary_cross_entropy = 0.864527
Epoch 242
Validation binary_cross_entropy = 0.836601
Epoch 243
Validation binary_cross_entropy = 0.824363
Epoch 244
Loss = 1.0365e-01, PNorm = 67.7502, GNorm = 1.5948, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 0.816309
Epoch 245
Validation binary_cross_entropy = 0.805920
Epoch 246
Validation binary_cross_entropy = 0.816564
Epoch 247
Validation binary_cross_entropy = 0.832855
Epoch 248
Validation binary_cross_entropy = 0.839924
Epoch 249
Loss = 9.9414e-02, PNorm = 67.8105, GNorm = 0.8862, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 0.858812
Epoch 250
Validation binary_cross_entropy = 0.884752
Epoch 251
Validation binary_cross_entropy = 0.943205
Epoch 252
Validation binary_cross_entropy = 0.871087
Epoch 253
Validation binary_cross_entropy = 0.843792
Epoch 254
Loss = 5.8441e-02, PNorm = 67.8776, GNorm = 1.2369, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 0.834478
Epoch 255
Validation binary_cross_entropy = 0.823732
Epoch 256
Validation binary_cross_entropy = 0.827302
Epoch 257
Validation binary_cross_entropy = 0.841992
Epoch 258
Validation binary_cross_entropy = 0.844975
Epoch 259
Loss = 3.9627e-02, PNorm = 67.9412, GNorm = 1.5323, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 0.861016
Epoch 260
Validation binary_cross_entropy = 0.891484
Epoch 261
Validation binary_cross_entropy = 0.930675
Epoch 262
Validation binary_cross_entropy = 0.976460
Epoch 263
Validation binary_cross_entropy = 1.013079
Epoch 264
Loss = 2.5555e-02, PNorm = 67.9924, GNorm = 0.7473, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 1.048298
Epoch 265
Validation binary_cross_entropy = 1.085888
Epoch 266
Validation binary_cross_entropy = 1.136242
Epoch 267
Validation binary_cross_entropy = 1.138663
Epoch 268
Validation binary_cross_entropy = 1.130452
Epoch 269
Loss = 9.2432e-02, PNorm = 68.0279, GNorm = 0.8972, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 1.149886
Epoch 270
Validation binary_cross_entropy = 1.094458
Epoch 271
Validation binary_cross_entropy = 1.043505
Epoch 272
Validation binary_cross_entropy = 1.029505
Epoch 273
Validation binary_cross_entropy = 1.020263
Epoch 274
Loss = 5.9578e-02, PNorm = 68.0705, GNorm = 2.8084, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 1.024821
Epoch 275
Validation binary_cross_entropy = 1.015292
Epoch 276
Validation binary_cross_entropy = 1.076390
Epoch 277
Validation binary_cross_entropy = 1.029383
Epoch 278
Validation binary_cross_entropy = 0.931234
Epoch 279
Loss = 7.1647e-02, PNorm = 68.1198, GNorm = 1.1033, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 0.881514
Epoch 280
Validation binary_cross_entropy = 0.879637
Epoch 281
Validation binary_cross_entropy = 0.899193
Epoch 282
Validation binary_cross_entropy = 0.932059
Epoch 283
Validation binary_cross_entropy = 1.008843
Epoch 284
Loss = 8.2893e-02, PNorm = 68.1745, GNorm = 1.4582, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 1.105471
Epoch 285
Validation binary_cross_entropy = 1.096739
Epoch 286
Validation binary_cross_entropy = 1.113582
Epoch 287
Validation binary_cross_entropy = 1.121947
Epoch 288
Validation binary_cross_entropy = 1.177566
Epoch 289
Loss = 2.7953e-02, PNorm = 68.2217, GNorm = 2.0306, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 1.314124
Epoch 290
Validation binary_cross_entropy = 1.178311
Epoch 291
Validation binary_cross_entropy = 1.073959
Epoch 292
Validation binary_cross_entropy = 1.002623
Epoch 293
Validation binary_cross_entropy = 0.782546
Epoch 294
Loss = 2.4711e-01, PNorm = 68.2811, GNorm = 1.2912, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 0.702418
Epoch 295
Validation binary_cross_entropy = 0.641909
Epoch 296
Validation binary_cross_entropy = 0.570542
Epoch 297
Validation binary_cross_entropy = 0.546948
Epoch 298
Validation binary_cross_entropy = 0.553379
Epoch 299
Loss = 1.2045e-01, PNorm = 68.3430, GNorm = 0.9221, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 0.574303
Model 0 best validation binary_cross_entropy = 0.270032 on epoch 81
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.205699
Ensemble test binary_cross_entropy = 0.205699
Fold 3
Splitting data with seed 3
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.4, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=800, bias=False)
        (W_h): Linear(in_features=800, out_features=800, bias=False)
        (W_o): Linear(in_features=933, out_features=800, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.4, inplace=False)
    (1): Linear(in_features=3100, out_features=800, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.4, inplace=False)
    (4): Linear(in_features=800, out_features=800, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.4, inplace=False)
    (7): Linear(in_features=800, out_features=1, bias=True)
  )
)
Number of parameters = 4,627,201
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 0.292552
Epoch 1
Validation binary_cross_entropy = 0.314568
Epoch 2
Validation binary_cross_entropy = 0.721326
Epoch 3
Validation binary_cross_entropy = 0.305099
Epoch 4
Loss = 8.1420e-01, PNorm = 63.1529, GNorm = 6.9891, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 0.290179
Epoch 5
Validation binary_cross_entropy = 0.581570
Epoch 6
Validation binary_cross_entropy = 0.592783
Epoch 7
Validation binary_cross_entropy = 0.288288
Epoch 8
Validation binary_cross_entropy = 0.323829
Epoch 9
Loss = 6.8788e-01, PNorm = 63.2945, GNorm = 4.8888, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 0.737028
Epoch 10
Validation binary_cross_entropy = 0.332306
Epoch 11
Validation binary_cross_entropy = 0.324212
Epoch 12
Validation binary_cross_entropy = 0.496599
Epoch 13
Validation binary_cross_entropy = 0.373763
Epoch 14
Loss = 7.4570e-01, PNorm = 63.4460, GNorm = 4.5970, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 0.318944
Epoch 15
Validation binary_cross_entropy = 0.560535
Epoch 16
Validation binary_cross_entropy = 0.523742
Epoch 17
Validation binary_cross_entropy = 0.319813
Epoch 18
Validation binary_cross_entropy = 0.371191
Epoch 19
Loss = 7.1945e-01, PNorm = 63.5611, GNorm = 3.8456, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 0.643111
Epoch 20
Validation binary_cross_entropy = 0.409281
Epoch 21
Validation binary_cross_entropy = 0.307546
Epoch 22
Validation binary_cross_entropy = 0.365077
Epoch 23
Validation binary_cross_entropy = 0.621935
Epoch 24
Loss = 6.8321e-01, PNorm = 63.6446, GNorm = 4.3268, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 0.424910
Epoch 25
Validation binary_cross_entropy = 0.333438
Epoch 26
Validation binary_cross_entropy = 0.416331
Epoch 27
Validation binary_cross_entropy = 0.688589
Epoch 28
Validation binary_cross_entropy = 0.508631
Epoch 29
Loss = 5.8375e-01, PNorm = 63.7217, GNorm = 2.2870, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 0.449368
Epoch 30
Validation binary_cross_entropy = 0.498492
Epoch 31
Validation binary_cross_entropy = 0.588410
Epoch 32
Validation binary_cross_entropy = 0.406770
Epoch 33
Validation binary_cross_entropy = 0.467535
Epoch 34
Loss = 6.7340e-01, PNorm = 63.7850, GNorm = 2.9855, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 0.831251
Epoch 35
Validation binary_cross_entropy = 0.867948
Epoch 36
Validation binary_cross_entropy = 0.526878
Epoch 37
Validation binary_cross_entropy = 0.382611
Epoch 38
Validation binary_cross_entropy = 0.437268
Epoch 39
Loss = 6.4439e-01, PNorm = 63.8663, GNorm = 2.9600, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 0.646669
Epoch 40
Validation binary_cross_entropy = 0.688491
Epoch 41
Validation binary_cross_entropy = 0.555366
Epoch 42
Validation binary_cross_entropy = 0.426560
Epoch 43
Validation binary_cross_entropy = 0.391114
Epoch 44
Loss = 6.4086e-01, PNorm = 63.9220, GNorm = 2.3617, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 0.460353
Epoch 45
Validation binary_cross_entropy = 0.567913
Epoch 46
Validation binary_cross_entropy = 0.612058
Epoch 47
Validation binary_cross_entropy = 0.529652
Epoch 48
Validation binary_cross_entropy = 0.482848
Epoch 49
Loss = 5.5242e-01, PNorm = 63.9805, GNorm = 1.8685, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 0.466197
Epoch 50
Validation binary_cross_entropy = 0.488856
Epoch 51
Validation binary_cross_entropy = 0.517385
Epoch 52
Validation binary_cross_entropy = 0.512792
Epoch 53
Validation binary_cross_entropy = 0.483654
Epoch 54
Loss = 5.3956e-01, PNorm = 64.0228, GNorm = 1.8113, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 0.495857
Epoch 55
Validation binary_cross_entropy = 0.578116
Epoch 56
Validation binary_cross_entropy = 0.611870
Epoch 57
Validation binary_cross_entropy = 0.549744
Epoch 58
Validation binary_cross_entropy = 0.485417
Epoch 59
Loss = 6.3925e-01, PNorm = 64.0839, GNorm = 1.9920, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 0.446963
Epoch 60
Validation binary_cross_entropy = 0.538598
Epoch 61
Validation binary_cross_entropy = 0.583346
Epoch 62
Validation binary_cross_entropy = 0.561089
Epoch 63
Validation binary_cross_entropy = 0.496523
Epoch 64
Loss = 5.6670e-01, PNorm = 64.1440, GNorm = 1.5366, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 0.472133
Epoch 65
Validation binary_cross_entropy = 0.485509
Epoch 66
Validation binary_cross_entropy = 0.524734
Epoch 67
Validation binary_cross_entropy = 0.580696
Epoch 68
Validation binary_cross_entropy = 0.499647
Epoch 69
Loss = 5.4936e-01, PNorm = 64.2028, GNorm = 1.5289, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 0.460163
Epoch 70
Validation binary_cross_entropy = 0.507534
Epoch 71
Validation binary_cross_entropy = 0.564477
Epoch 72
Validation binary_cross_entropy = 0.439626
Epoch 73
Validation binary_cross_entropy = 0.401142
Epoch 74
Loss = 5.3582e-01, PNorm = 64.2652, GNorm = 1.6479, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 0.489256
Epoch 75
Validation binary_cross_entropy = 0.612832
Epoch 76
Validation binary_cross_entropy = 0.634315
Epoch 77
Validation binary_cross_entropy = 0.465726
Epoch 78
Validation binary_cross_entropy = 0.378660
Epoch 79
Loss = 5.7250e-01, PNorm = 64.3428, GNorm = 2.6554, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 0.432455
Epoch 80
Validation binary_cross_entropy = 0.528566
Epoch 81
Validation binary_cross_entropy = 0.494160
Epoch 82
Validation binary_cross_entropy = 0.389119
Epoch 83
Validation binary_cross_entropy = 0.386911
Epoch 84
Loss = 4.2573e-01, PNorm = 64.4444, GNorm = 1.8711, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 0.626095
Epoch 85
Validation binary_cross_entropy = 0.357479
Epoch 86
Validation binary_cross_entropy = 0.468645
Epoch 87
Validation binary_cross_entropy = 0.396168
Epoch 88
Validation binary_cross_entropy = 0.541328
Epoch 89
Loss = 5.1210e-01, PNorm = 64.5409, GNorm = 2.1628, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 0.432208
Epoch 90
Validation binary_cross_entropy = 0.350868
Epoch 91
Validation binary_cross_entropy = 0.405249
Epoch 92
Validation binary_cross_entropy = 0.386058
Epoch 93
Validation binary_cross_entropy = 0.354566
Epoch 94
Loss = 4.3531e-01, PNorm = 64.6587, GNorm = 1.8769, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 0.406523
Epoch 95
Validation binary_cross_entropy = 0.481656
Epoch 96
Validation binary_cross_entropy = 0.307152
Epoch 97
Validation binary_cross_entropy = 0.415882
Epoch 98
Validation binary_cross_entropy = 0.470450
Epoch 99
Loss = 3.9036e-01, PNorm = 64.7764, GNorm = 1.3740, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.388397
Epoch 100
Validation binary_cross_entropy = 0.388972
Epoch 101
Validation binary_cross_entropy = 0.426312
Epoch 102
Validation binary_cross_entropy = 0.335682
Epoch 103
Validation binary_cross_entropy = 0.362589
Epoch 104
Loss = 4.7210e-01, PNorm = 64.9009, GNorm = 2.5020, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.367900
Epoch 105
Validation binary_cross_entropy = 0.415055
Epoch 106
Validation binary_cross_entropy = 0.353359
Epoch 107
Validation binary_cross_entropy = 0.378082
Epoch 108
Validation binary_cross_entropy = 0.394396
Epoch 109
Loss = 4.9544e-01, PNorm = 65.0290, GNorm = 4.3292, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.327714
Epoch 110
Validation binary_cross_entropy = 0.533865
Epoch 111
Validation binary_cross_entropy = 0.397293
Epoch 112
Validation binary_cross_entropy = 0.339235
Epoch 113
Validation binary_cross_entropy = 0.558443
Epoch 114
Loss = 5.1693e-01, PNorm = 65.1670, GNorm = 2.9845, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.440673
Epoch 115
Validation binary_cross_entropy = 0.307551
Epoch 116
Validation binary_cross_entropy = 0.482742
Epoch 117
Validation binary_cross_entropy = 0.441121
Epoch 118
Validation binary_cross_entropy = 0.349394
Epoch 119
Loss = 3.4035e-01, PNorm = 65.3126, GNorm = 1.3693, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.366206
Epoch 120
Validation binary_cross_entropy = 0.379846
Epoch 121
Validation binary_cross_entropy = 0.405693
Epoch 122
Validation binary_cross_entropy = 0.413864
Epoch 123
Validation binary_cross_entropy = 0.425533
Epoch 124
Loss = 3.2348e-01, PNorm = 65.4496, GNorm = 2.9674, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.547722
Epoch 125
Validation binary_cross_entropy = 0.355873
Epoch 126
Validation binary_cross_entropy = 0.392602
Epoch 127
Validation binary_cross_entropy = 0.661419
Epoch 128
Validation binary_cross_entropy = 0.368440
Epoch 129
Loss = 3.6698e-01, PNorm = 65.5949, GNorm = 1.4752, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.352750
Epoch 130
Validation binary_cross_entropy = 0.411927
Epoch 131
Validation binary_cross_entropy = 0.481080
Epoch 132
Validation binary_cross_entropy = 0.402232
Epoch 133
Validation binary_cross_entropy = 0.402027
Epoch 134
Loss = 3.8293e-01, PNorm = 65.7560, GNorm = 2.7578, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.491551
Epoch 135
Validation binary_cross_entropy = 0.410273
Epoch 136
Validation binary_cross_entropy = 0.397548
Epoch 137
Validation binary_cross_entropy = 0.445451
Epoch 138
Validation binary_cross_entropy = 0.508258
Epoch 139
Loss = 2.0537e-01, PNorm = 65.8864, GNorm = 1.0507, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.481198
Epoch 140
Validation binary_cross_entropy = 0.479820
Epoch 141
Validation binary_cross_entropy = 0.526502
Epoch 142
Validation binary_cross_entropy = 0.574982
Epoch 143
Validation binary_cross_entropy = 0.612215
Epoch 144
Loss = 1.4681e-01, PNorm = 66.0114, GNorm = 2.1856, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.639077
Epoch 145
Validation binary_cross_entropy = 0.698407
Epoch 146
Validation binary_cross_entropy = 0.632611
Epoch 147
Validation binary_cross_entropy = 0.594927
Epoch 148
Validation binary_cross_entropy = 0.561614
Epoch 149
Loss = 2.0230e-01, PNorm = 66.1266, GNorm = 2.8641, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.585597
Epoch 150
Validation binary_cross_entropy = 0.540583
Epoch 151
Validation binary_cross_entropy = 0.558106
Epoch 152
Validation binary_cross_entropy = 0.586040
Epoch 153
Validation binary_cross_entropy = 0.586533
Epoch 154
Loss = 1.6963e-01, PNorm = 66.2396, GNorm = 1.3229, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.604818
Epoch 155
Validation binary_cross_entropy = 0.590216
Epoch 156
Validation binary_cross_entropy = 0.615600
Epoch 157
Validation binary_cross_entropy = 0.632788
Epoch 158
Validation binary_cross_entropy = 0.650601
Epoch 159
Loss = 2.1232e-01, PNorm = 66.3510, GNorm = 1.5713, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.663122
Epoch 160
Validation binary_cross_entropy = 0.651727
Epoch 161
Validation binary_cross_entropy = 0.442205
Epoch 162
Validation binary_cross_entropy = 0.394072
Epoch 163
Validation binary_cross_entropy = 0.399031
Epoch 164
Loss = 2.4766e-01, PNorm = 66.4620, GNorm = 1.4658, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.440432
Epoch 165
Validation binary_cross_entropy = 0.502177
Epoch 166
Validation binary_cross_entropy = 0.531155
Epoch 167
Validation binary_cross_entropy = 0.539069
Epoch 168
Validation binary_cross_entropy = 0.534165
Epoch 169
Loss = 1.5560e-01, PNorm = 66.5774, GNorm = 1.6666, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.524106
Epoch 170
Validation binary_cross_entropy = 0.523989
Epoch 171
Validation binary_cross_entropy = 0.541298
Epoch 172
Validation binary_cross_entropy = 0.577537
Epoch 173
Validation binary_cross_entropy = 0.620675
Epoch 174
Loss = 1.9123e-01, PNorm = 66.6709, GNorm = 1.7372, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 0.647582
Epoch 175
Validation binary_cross_entropy = 0.675860
Epoch 176
Validation binary_cross_entropy = 0.667419
Epoch 177
Validation binary_cross_entropy = 0.670002
Epoch 178
Validation binary_cross_entropy = 0.691040
Epoch 179
Loss = 1.5681e-01, PNorm = 66.7594, GNorm = 6.3627, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 0.679647
Epoch 180
Validation binary_cross_entropy = 0.678242
Epoch 181
Validation binary_cross_entropy = 0.640047
Epoch 182
Validation binary_cross_entropy = 0.621404
Epoch 183
Validation binary_cross_entropy = 0.580613
Epoch 184
Loss = 9.1329e-02, PNorm = 66.8666, GNorm = 2.6225, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 0.620638
Epoch 185
Validation binary_cross_entropy = 0.618826
Epoch 186
Validation binary_cross_entropy = 0.665282
Epoch 187
Validation binary_cross_entropy = 0.700099
Epoch 188
Validation binary_cross_entropy = 0.702531
Epoch 189
Loss = 6.4512e-02, PNorm = 66.9703, GNorm = 1.2093, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 0.732174
Epoch 190
Validation binary_cross_entropy = 0.732423
Epoch 191
Validation binary_cross_entropy = 0.754431
Epoch 192
Validation binary_cross_entropy = 0.707307
Epoch 193
Validation binary_cross_entropy = 0.689016
Epoch 194
Loss = 1.0651e-01, PNorm = 67.0517, GNorm = 1.0455, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 0.663620
Epoch 195
Validation binary_cross_entropy = 0.643563
Epoch 196
Validation binary_cross_entropy = 0.656336
Epoch 197
Validation binary_cross_entropy = 0.659149
Epoch 198
Validation binary_cross_entropy = 0.628559
Epoch 199
Loss = 7.2480e-02, PNorm = 67.1227, GNorm = 0.9048, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 0.639900
Epoch 200
Validation binary_cross_entropy = 0.670813
Epoch 201
Validation binary_cross_entropy = 0.733165
Epoch 202
Validation binary_cross_entropy = 0.720059
Epoch 203
Validation binary_cross_entropy = 0.740153
Epoch 204
Loss = 3.7456e-02, PNorm = 67.2031, GNorm = 0.2152, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 0.755571
Epoch 205
Validation binary_cross_entropy = 0.781123
Epoch 206
Validation binary_cross_entropy = 0.825172
Epoch 207
Validation binary_cross_entropy = 0.877504
Epoch 208
Validation binary_cross_entropy = 0.922026
Epoch 209
Loss = 1.3644e-01, PNorm = 67.2714, GNorm = 1.5548, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 0.934244
Epoch 210
Validation binary_cross_entropy = 0.936847
Epoch 211
Validation binary_cross_entropy = 0.930200
Epoch 212
Validation binary_cross_entropy = 0.921687
Epoch 213
Validation binary_cross_entropy = 0.903498
Epoch 214
Loss = 1.7617e-01, PNorm = 67.3364, GNorm = 3.6271, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 0.873738
Epoch 215
Validation binary_cross_entropy = 0.878439
Epoch 216
Validation binary_cross_entropy = 0.836911
Epoch 217
Validation binary_cross_entropy = 0.818574
Epoch 218
Validation binary_cross_entropy = 0.836896
Epoch 219
Loss = 8.2579e-02, PNorm = 67.4039, GNorm = 1.1818, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 0.870595
Epoch 220
Validation binary_cross_entropy = 0.914395
Epoch 221
Validation binary_cross_entropy = 0.913149
Epoch 222
Validation binary_cross_entropy = 0.913756
Epoch 223
Validation binary_cross_entropy = 0.924241
Epoch 224
Loss = 6.3227e-02, PNorm = 67.4713, GNorm = 3.7481, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 0.960573
Epoch 225
Validation binary_cross_entropy = 0.996536
Epoch 226
Validation binary_cross_entropy = 1.012333
Epoch 227
Validation binary_cross_entropy = 1.016157
Epoch 228
Validation binary_cross_entropy = 1.003622
Epoch 229
Loss = 1.0721e-01, PNorm = 67.5428, GNorm = 0.8506, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 1.035002
Epoch 230
Validation binary_cross_entropy = 0.956165
Epoch 231
Validation binary_cross_entropy = 0.962658
Epoch 232
Validation binary_cross_entropy = 0.968113
Epoch 233
Validation binary_cross_entropy = 0.938806
Epoch 234
Loss = 1.6827e-01, PNorm = 67.6229, GNorm = 2.1261, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 0.975307
Epoch 235
Validation binary_cross_entropy = 0.805311
Epoch 236
Validation binary_cross_entropy = 0.753996
Epoch 237
Validation binary_cross_entropy = 0.724742
Epoch 238
Validation binary_cross_entropy = 0.699178
Epoch 239
Loss = 1.7107e-01, PNorm = 67.7074, GNorm = 2.1629, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 0.739800
Epoch 240
Validation binary_cross_entropy = 0.789324
Epoch 241
Validation binary_cross_entropy = 0.750819
Epoch 242
Validation binary_cross_entropy = 0.738809
Epoch 243
Validation binary_cross_entropy = 0.745816
Epoch 244
Loss = 1.6517e-01, PNorm = 67.8062, GNorm = 1.3805, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 0.747619
Epoch 245
Validation binary_cross_entropy = 0.760351
Epoch 246
Validation binary_cross_entropy = 0.795803
Epoch 247
Validation binary_cross_entropy = 0.843399
Epoch 248
Validation binary_cross_entropy = 0.902029
Epoch 249
Loss = 1.5845e-01, PNorm = 67.8743, GNorm = 5.7840, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 0.885994
Epoch 250
Validation binary_cross_entropy = 0.837175
Epoch 251
Validation binary_cross_entropy = 0.834124
Epoch 252
Validation binary_cross_entropy = 0.863985
Epoch 253
Validation binary_cross_entropy = 0.918563
Epoch 254
Loss = 1.6137e-01, PNorm = 67.9331, GNorm = 1.0385, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 0.823608
Epoch 255
Validation binary_cross_entropy = 0.778850
Epoch 256
Validation binary_cross_entropy = 0.781743
Epoch 257
Validation binary_cross_entropy = 0.801988
Epoch 258
Validation binary_cross_entropy = 0.872942
Epoch 259
Loss = 1.3978e-01, PNorm = 67.9915, GNorm = 2.4525, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 0.981546
Epoch 260
Validation binary_cross_entropy = 1.006060
Epoch 261
Validation binary_cross_entropy = 1.036832
Epoch 262
Validation binary_cross_entropy = 1.081043
Epoch 263
Validation binary_cross_entropy = 1.143172
Epoch 264
Loss = 5.1914e-02, PNorm = 68.0605, GNorm = 1.4255, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 1.174832
Epoch 265
Validation binary_cross_entropy = 1.150185
Epoch 266
Validation binary_cross_entropy = 1.142175
Epoch 267
Validation binary_cross_entropy = 1.163554
Epoch 268
Validation binary_cross_entropy = 1.149454
Epoch 269
Loss = 3.8407e-02, PNorm = 68.1108, GNorm = 0.9273, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 1.114847
Epoch 270
Validation binary_cross_entropy = 1.091699
Epoch 271
Validation binary_cross_entropy = 1.082084
Epoch 272
Validation binary_cross_entropy = 1.086115
Epoch 273
Validation binary_cross_entropy = 1.058876
Epoch 274
Loss = 4.1368e-02, PNorm = 68.1609, GNorm = 0.8233, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 1.024378
Epoch 275
Validation binary_cross_entropy = 1.022900
Epoch 276
Validation binary_cross_entropy = 1.059739
Epoch 277
Validation binary_cross_entropy = 1.141862
Epoch 278
Validation binary_cross_entropy = 1.157985
Epoch 279
Loss = 4.0875e-02, PNorm = 68.2106, GNorm = 0.2399, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 1.032394
Epoch 280
Validation binary_cross_entropy = 1.007251
Epoch 281
Validation binary_cross_entropy = 1.044358
Epoch 282
Validation binary_cross_entropy = 1.103137
Epoch 283
Validation binary_cross_entropy = 1.031992
Epoch 284
Loss = 9.1577e-02, PNorm = 68.2593, GNorm = 0.5152, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 0.996821
Epoch 285
Validation binary_cross_entropy = 1.025843
Epoch 286
Validation binary_cross_entropy = 1.102168
Epoch 287
Validation binary_cross_entropy = 1.088647
Epoch 288
Validation binary_cross_entropy = 1.096987
Epoch 289
Loss = 6.6537e-02, PNorm = 68.3209, GNorm = 1.1367, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 1.127356
Epoch 290
Validation binary_cross_entropy = 1.191445
Epoch 291
Validation binary_cross_entropy = 1.210304
Epoch 292
Validation binary_cross_entropy = 1.259330
Epoch 293
Validation binary_cross_entropy = 1.323222
Epoch 294
Loss = 5.5544e-02, PNorm = 68.3817, GNorm = 1.4752, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 1.327587
Epoch 295
Validation binary_cross_entropy = 1.307706
Epoch 296
Validation binary_cross_entropy = 1.311044
Epoch 297
Validation binary_cross_entropy = 1.358216
Epoch 298
Validation binary_cross_entropy = 1.458270
Epoch 299
Loss = 9.2846e-02, PNorm = 68.4349, GNorm = 1.0045, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 1.349580
Model 0 best validation binary_cross_entropy = 0.288288 on epoch 7
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.244754
Ensemble test binary_cross_entropy = 0.244754
Fold 4
Splitting data with seed 4
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.4, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=800, bias=False)
        (W_h): Linear(in_features=800, out_features=800, bias=False)
        (W_o): Linear(in_features=933, out_features=800, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.4, inplace=False)
    (1): Linear(in_features=3100, out_features=800, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.4, inplace=False)
    (4): Linear(in_features=800, out_features=800, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.4, inplace=False)
    (7): Linear(in_features=800, out_features=1, bias=True)
  )
)
Number of parameters = 4,627,201
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 0.288492
Epoch 1
Validation binary_cross_entropy = 0.299842
Epoch 2
Validation binary_cross_entropy = 0.543299
Epoch 3
Validation binary_cross_entropy = 0.315291
Epoch 4
Loss = 8.1265e-01, PNorm = 63.1515, GNorm = 7.8907, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 0.296001
Epoch 5
Validation binary_cross_entropy = 0.422690
Epoch 6
Validation binary_cross_entropy = 0.719525
Epoch 7
Validation binary_cross_entropy = 0.306277
Epoch 8
Validation binary_cross_entropy = 0.396395
Epoch 9
Loss = 6.9810e-01, PNorm = 63.2896, GNorm = 6.6396, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 0.909441
Epoch 10
Validation binary_cross_entropy = 0.319208
Epoch 11
Validation binary_cross_entropy = 0.344195
Epoch 12
Validation binary_cross_entropy = 0.870440
Epoch 13
Validation binary_cross_entropy = 0.701116
Epoch 14
Loss = 8.1211e-01, PNorm = 63.4412, GNorm = 4.5585, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 0.306238
Epoch 15
Validation binary_cross_entropy = 0.491329
Epoch 16
Validation binary_cross_entropy = 0.811347
Epoch 17
Validation binary_cross_entropy = 0.366089
Epoch 18
Validation binary_cross_entropy = 0.339635
Epoch 19
Loss = 7.1207e-01, PNorm = 63.5675, GNorm = 3.5811, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 0.571972
Epoch 20
Validation binary_cross_entropy = 0.852517
Epoch 21
Validation binary_cross_entropy = 0.555954
Epoch 22
Validation binary_cross_entropy = 0.412846
Epoch 23
Validation binary_cross_entropy = 0.692608
Epoch 24
Loss = 6.7237e-01, PNorm = 63.6636, GNorm = 3.9711, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 0.614873
Epoch 25
Validation binary_cross_entropy = 0.441450
Epoch 26
Validation binary_cross_entropy = 0.397073
Epoch 27
Validation binary_cross_entropy = 0.483526
Epoch 28
Validation binary_cross_entropy = 0.552853
Epoch 29
Loss = 5.9715e-01, PNorm = 63.7372, GNorm = 2.5620, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 0.477576
Epoch 30
Validation binary_cross_entropy = 0.450970
Epoch 31
Validation binary_cross_entropy = 0.475460
Epoch 32
Validation binary_cross_entropy = 0.556908
Epoch 33
Validation binary_cross_entropy = 0.406190
Epoch 34
Loss = 6.0678e-01, PNorm = 63.8067, GNorm = 2.5643, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 0.381212
Epoch 35
Validation binary_cross_entropy = 0.412417
Epoch 36
Validation binary_cross_entropy = 0.497742
Epoch 37
Validation binary_cross_entropy = 0.578227
Epoch 38
Validation binary_cross_entropy = 0.503798
Epoch 39
Loss = 7.0062e-01, PNorm = 63.8593, GNorm = 3.6762, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 0.410573
Epoch 40
Validation binary_cross_entropy = 0.462313
Epoch 41
Validation binary_cross_entropy = 0.612047
Epoch 42
Validation binary_cross_entropy = 0.558900
Epoch 43
Validation binary_cross_entropy = 0.401490
Epoch 44
Loss = 5.9761e-01, PNorm = 63.9216, GNorm = 2.6747, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 0.397668
Epoch 45
Validation binary_cross_entropy = 0.490424
Epoch 46
Validation binary_cross_entropy = 0.620691
Epoch 47
Validation binary_cross_entropy = 0.606092
Epoch 48
Validation binary_cross_entropy = 0.553369
Epoch 49
Loss = 5.9209e-01, PNorm = 63.9908, GNorm = 1.9154, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 0.463137
Epoch 50
Validation binary_cross_entropy = 0.449482
Epoch 51
Validation binary_cross_entropy = 0.484529
Epoch 52
Validation binary_cross_entropy = 0.498411
Epoch 53
Validation binary_cross_entropy = 0.439562
Epoch 54
Loss = 5.6853e-01, PNorm = 64.0589, GNorm = 1.9765, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 0.428338
Epoch 55
Validation binary_cross_entropy = 0.488247
Epoch 56
Validation binary_cross_entropy = 0.557612
Epoch 57
Validation binary_cross_entropy = 0.533687
Epoch 58
Validation binary_cross_entropy = 0.515251
Epoch 59
Loss = 6.5696e-01, PNorm = 64.1299, GNorm = 2.6334, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 0.419050
Epoch 60
Validation binary_cross_entropy = 0.417131
Epoch 61
Validation binary_cross_entropy = 0.467819
Epoch 62
Validation binary_cross_entropy = 0.473204
Epoch 63
Validation binary_cross_entropy = 0.488440
Epoch 64
Loss = 5.2813e-01, PNorm = 64.2105, GNorm = 1.6724, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 0.547683
Epoch 65
Validation binary_cross_entropy = 0.401022
Epoch 66
Validation binary_cross_entropy = 0.297535
Epoch 67
Validation binary_cross_entropy = 0.409309
Epoch 68
Validation binary_cross_entropy = 0.669960
Epoch 69
Loss = 5.9968e-01, PNorm = 64.2814, GNorm = 2.0931, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 0.589756
Epoch 70
Validation binary_cross_entropy = 0.451441
Epoch 71
Validation binary_cross_entropy = 0.416110
Epoch 72
Validation binary_cross_entropy = 0.482423
Epoch 73
Validation binary_cross_entropy = 0.553170
Epoch 74
Loss = 5.6295e-01, PNorm = 64.3738, GNorm = 1.3523, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 0.431929
Epoch 75
Validation binary_cross_entropy = 0.379163
Epoch 76
Validation binary_cross_entropy = 0.405960
Epoch 77
Validation binary_cross_entropy = 0.412164
Epoch 78
Validation binary_cross_entropy = 0.447813
Epoch 79
Loss = 5.2126e-01, PNorm = 64.4663, GNorm = 2.0992, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 0.393825
Epoch 80
Validation binary_cross_entropy = 0.369460
Epoch 81
Validation binary_cross_entropy = 0.486076
Epoch 82
Validation binary_cross_entropy = 0.467802
Epoch 83
Validation binary_cross_entropy = 0.370557
Epoch 84
Loss = 4.3767e-01, PNorm = 64.5614, GNorm = 1.5679, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 0.472103
Epoch 85
Validation binary_cross_entropy = 0.459200
Epoch 86
Validation binary_cross_entropy = 0.339749
Epoch 87
Validation binary_cross_entropy = 0.542451
Epoch 88
Validation binary_cross_entropy = 0.441087
Epoch 89
Loss = 4.1964e-01, PNorm = 64.6586, GNorm = 2.0495, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 0.319750
Epoch 90
Validation binary_cross_entropy = 0.373408
Epoch 91
Validation binary_cross_entropy = 0.502973
Epoch 92
Validation binary_cross_entropy = 0.405448
Epoch 93
Validation binary_cross_entropy = 0.317231
Epoch 94
Loss = 4.8523e-01, PNorm = 64.7657, GNorm = 1.4608, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 0.420130
Epoch 95
Validation binary_cross_entropy = 0.429675
Epoch 96
Validation binary_cross_entropy = 0.319749
Epoch 97
Validation binary_cross_entropy = 0.380187
Epoch 98
Validation binary_cross_entropy = 0.307637
Epoch 99
Loss = 3.4253e-01, PNorm = 64.8643, GNorm = 2.1386, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.466270
Epoch 100
Validation binary_cross_entropy = 0.345845
Epoch 101
Validation binary_cross_entropy = 0.288632
Epoch 102
Validation binary_cross_entropy = 0.473014
Epoch 103
Validation binary_cross_entropy = 0.397956
Epoch 104
Loss = 3.2407e-01, PNorm = 64.9747, GNorm = 1.5386, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.344894
Epoch 105
Validation binary_cross_entropy = 0.539123
Epoch 106
Validation binary_cross_entropy = 0.433450
Epoch 107
Validation binary_cross_entropy = 0.368778
Epoch 108
Validation binary_cross_entropy = 0.463620
Epoch 109
Loss = 3.0781e-01, PNorm = 65.1021, GNorm = 2.7088, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.372240
Epoch 110
Validation binary_cross_entropy = 0.395180
Epoch 111
Validation binary_cross_entropy = 0.418839
Epoch 112
Validation binary_cross_entropy = 0.319872
Epoch 113
Validation binary_cross_entropy = 0.435981
Epoch 114
Loss = 3.5858e-01, PNorm = 65.2264, GNorm = 3.6772, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.358717
Epoch 115
Validation binary_cross_entropy = 0.303516
Epoch 116
Validation binary_cross_entropy = 0.335892
Epoch 117
Validation binary_cross_entropy = 0.413988
Epoch 118
Validation binary_cross_entropy = 0.425450
Epoch 119
Loss = 4.0860e-01, PNorm = 65.3553, GNorm = 2.6665, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.348972
Epoch 120
Validation binary_cross_entropy = 0.601076
Epoch 121
Validation binary_cross_entropy = 0.320109
Epoch 122
Validation binary_cross_entropy = 0.287189
Epoch 123
Validation binary_cross_entropy = 0.365995
Epoch 124
Loss = 3.1365e-01, PNorm = 65.4845, GNorm = 1.5758, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.553133
Epoch 125
Validation binary_cross_entropy = 0.504318
Epoch 126
Validation binary_cross_entropy = 0.347122
Epoch 127
Validation binary_cross_entropy = 0.374508
Epoch 128
Validation binary_cross_entropy = 0.486244
Epoch 129
Loss = 2.6749e-01, PNorm = 65.6170, GNorm = 2.7061, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.451229
Epoch 130
Validation binary_cross_entropy = 0.361152
Epoch 131
Validation binary_cross_entropy = 0.363787
Epoch 132
Validation binary_cross_entropy = 0.408224
Epoch 133
Validation binary_cross_entropy = 0.421603
Epoch 134
Loss = 2.5335e-01, PNorm = 65.7448, GNorm = 1.2757, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.411673
Epoch 135
Validation binary_cross_entropy = 0.432730
Epoch 136
Validation binary_cross_entropy = 0.468344
Epoch 137
Validation binary_cross_entropy = 0.480847
Epoch 138
Validation binary_cross_entropy = 0.549372
Epoch 139
Loss = 3.0719e-01, PNorm = 65.8600, GNorm = 3.6005, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.472428
Epoch 140
Validation binary_cross_entropy = 0.436960
Epoch 141
Validation binary_cross_entropy = 0.422572
Epoch 142
Validation binary_cross_entropy = 0.559141
Epoch 143
Validation binary_cross_entropy = 0.476101
Epoch 144
Loss = 1.5877e-01, PNorm = 65.9867, GNorm = 0.8794, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.457860
Epoch 145
Validation binary_cross_entropy = 0.493195
Epoch 146
Validation binary_cross_entropy = 0.514537
Epoch 147
Validation binary_cross_entropy = 0.564307
Epoch 148
Validation binary_cross_entropy = 0.577007
Epoch 149
Loss = 1.8379e-01, PNorm = 66.1100, GNorm = 4.5137, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.597148
Epoch 150
Validation binary_cross_entropy = 0.613608
Epoch 151
Validation binary_cross_entropy = 0.639884
Epoch 152
Validation binary_cross_entropy = 0.633445
Epoch 153
Validation binary_cross_entropy = 0.632491
Epoch 154
Loss = 1.1833e-01, PNorm = 66.2233, GNorm = 1.8283, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.649407
Epoch 155
Validation binary_cross_entropy = 0.676634
Epoch 156
Validation binary_cross_entropy = 0.710803
Epoch 157
Validation binary_cross_entropy = 0.708321
Epoch 158
Validation binary_cross_entropy = 0.680733
Epoch 159
Loss = 1.4231e-01, PNorm = 66.3165, GNorm = 4.4290, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.670964
Epoch 160
Validation binary_cross_entropy = 0.645692
Epoch 161
Validation binary_cross_entropy = 0.615526
Epoch 162
Validation binary_cross_entropy = 0.563780
Epoch 163
Validation binary_cross_entropy = 0.553024
Epoch 164
Loss = 2.4194e-01, PNorm = 66.4023, GNorm = 1.5600, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.494587
Epoch 165
Validation binary_cross_entropy = 0.501754
Epoch 166
Validation binary_cross_entropy = 0.497926
Epoch 167
Validation binary_cross_entropy = 0.507897
Epoch 168
Validation binary_cross_entropy = 0.542008
Epoch 169
Loss = 2.2428e-01, PNorm = 66.5016, GNorm = 1.3830, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.571778
Epoch 170
Validation binary_cross_entropy = 0.660411
Epoch 171
Validation binary_cross_entropy = 0.584957
Epoch 172
Validation binary_cross_entropy = 0.597132
Epoch 173
Validation binary_cross_entropy = 0.550206
Epoch 174
Loss = 1.2431e-01, PNorm = 66.5879, GNorm = 0.7695, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 0.601356
Epoch 175
Validation binary_cross_entropy = 0.627331
Epoch 176
Validation binary_cross_entropy = 0.572003
Epoch 177
Validation binary_cross_entropy = 0.595815
Epoch 178
Validation binary_cross_entropy = 0.602376
Epoch 179
Loss = 1.7807e-01, PNorm = 66.6700, GNorm = 1.1253, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 0.616359
Epoch 180
Validation binary_cross_entropy = 0.691526
Epoch 181
Validation binary_cross_entropy = 0.667546
Epoch 182
Validation binary_cross_entropy = 0.670233
Epoch 183
Validation binary_cross_entropy = 0.708248
Epoch 184
Loss = 1.2918e-01, PNorm = 66.7524, GNorm = 1.1723, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 0.763388
Epoch 185
Validation binary_cross_entropy = 0.787253
Epoch 186
Validation binary_cross_entropy = 0.796164
Epoch 187
Validation binary_cross_entropy = 0.846020
Epoch 188
Validation binary_cross_entropy = 0.888438
Epoch 189
Loss = 8.1052e-02, PNorm = 66.8272, GNorm = 1.2412, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 0.893048
Epoch 190
Validation binary_cross_entropy = 0.866334
Epoch 191
Validation binary_cross_entropy = 0.881472
Epoch 192
Validation binary_cross_entropy = 0.844109
Epoch 193
Validation binary_cross_entropy = 0.823337
Epoch 194
Loss = 1.7810e-01, PNorm = 66.8872, GNorm = 1.7229, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 0.738904
Epoch 195
Validation binary_cross_entropy = 0.698282
Epoch 196
Validation binary_cross_entropy = 0.697426
Epoch 197
Validation binary_cross_entropy = 0.688727
Epoch 198
Validation binary_cross_entropy = 0.647209
Epoch 199
Loss = 1.2401e-01, PNorm = 66.9585, GNorm = 1.4344, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 0.673292
Epoch 200
Validation binary_cross_entropy = 0.736361
Epoch 201
Validation binary_cross_entropy = 0.833612
Epoch 202
Validation binary_cross_entropy = 0.830946
Epoch 203
Validation binary_cross_entropy = 0.786475
Epoch 204
Loss = 6.5641e-02, PNorm = 67.0398, GNorm = 3.3544, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 0.812134
Epoch 205
Validation binary_cross_entropy = 0.881436
Epoch 206
Validation binary_cross_entropy = 0.866491
Epoch 207
Validation binary_cross_entropy = 0.842056
Epoch 208
Validation binary_cross_entropy = 0.800843
Epoch 209
Loss = 7.4235e-02, PNorm = 67.1174, GNorm = 1.1437, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 0.774655
Epoch 210
Validation binary_cross_entropy = 0.765011
Epoch 211
Validation binary_cross_entropy = 0.772774
Epoch 212
Validation binary_cross_entropy = 0.788674
Epoch 213
Validation binary_cross_entropy = 0.818022
Epoch 214
Loss = 1.6156e-01, PNorm = 67.1897, GNorm = 1.0422, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 0.860095
Epoch 215
Validation binary_cross_entropy = 0.887749
Epoch 216
Validation binary_cross_entropy = 0.879551
Epoch 217
Validation binary_cross_entropy = 0.952249
Epoch 218
Validation binary_cross_entropy = 0.940468
Epoch 219
Loss = 9.2942e-02, PNorm = 67.2484, GNorm = 2.2695, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 0.939447
Epoch 220
Validation binary_cross_entropy = 0.929503
Epoch 221
Validation binary_cross_entropy = 0.941944
Epoch 222
Validation binary_cross_entropy = 0.956086
Epoch 223
Validation binary_cross_entropy = 0.788120
Epoch 224
Loss = 5.3579e-02, PNorm = 67.3229, GNorm = 1.1557, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 0.711368
Epoch 225
Validation binary_cross_entropy = 0.696573
Epoch 226
Validation binary_cross_entropy = 0.707411
Epoch 227
Validation binary_cross_entropy = 0.650048
Epoch 228
Validation binary_cross_entropy = 0.618063
Epoch 229
Loss = 1.3217e-01, PNorm = 67.4078, GNorm = 1.2706, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 0.646415
Epoch 230
Validation binary_cross_entropy = 0.704910
Epoch 231
Validation binary_cross_entropy = 0.695859
Epoch 232
Validation binary_cross_entropy = 0.696799
Epoch 233
Validation binary_cross_entropy = 0.776171
Epoch 234
Loss = 5.0509e-02, PNorm = 67.4974, GNorm = 0.9569, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 0.886606
Epoch 235
Validation binary_cross_entropy = 0.952678
Epoch 236
Validation binary_cross_entropy = 0.862890
Epoch 237
Validation binary_cross_entropy = 0.851269
Epoch 238
Validation binary_cross_entropy = 0.834280
Epoch 239
Loss = 4.9108e-02, PNorm = 67.5672, GNorm = 0.2768, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 0.807954
Epoch 240
Validation binary_cross_entropy = 0.786730
Epoch 241
Validation binary_cross_entropy = 0.795628
Epoch 242
Validation binary_cross_entropy = 0.818009
Epoch 243
Validation binary_cross_entropy = 0.827724
Epoch 244
Loss = 5.5007e-02, PNorm = 67.6227, GNorm = 0.4361, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 0.828937
Epoch 245
Validation binary_cross_entropy = 0.841685
Epoch 246
Validation binary_cross_entropy = 0.915851
Epoch 247
Validation binary_cross_entropy = 0.939810
Epoch 248
Validation binary_cross_entropy = 0.875095
Epoch 249
Loss = 4.5072e-02, PNorm = 67.6748, GNorm = 2.0576, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 0.878501
Epoch 250
Validation binary_cross_entropy = 0.980757
Epoch 251
Validation binary_cross_entropy = 1.048447
Epoch 252
Validation binary_cross_entropy = 0.983834
Epoch 253
Validation binary_cross_entropy = 0.985679
Epoch 254
Loss = 4.4396e-02, PNorm = 67.7323, GNorm = 0.2994, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 1.050910
Epoch 255
Validation binary_cross_entropy = 1.091812
Epoch 256
Validation binary_cross_entropy = 1.088791
Epoch 257
Validation binary_cross_entropy = 1.028713
Epoch 258
Validation binary_cross_entropy = 1.025065
Epoch 259
Loss = 3.3006e-02, PNorm = 67.7895, GNorm = 0.7796, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 1.063460
Epoch 260
Validation binary_cross_entropy = 1.131845
Epoch 261
Validation binary_cross_entropy = 1.161702
Epoch 262
Validation binary_cross_entropy = 1.119502
Epoch 263
Validation binary_cross_entropy = 1.064125
Epoch 264
Loss = 1.2071e-01, PNorm = 67.8354, GNorm = 0.3764, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 0.972817
Epoch 265
Validation binary_cross_entropy = 0.998623
Epoch 266
Validation binary_cross_entropy = 1.004773
Epoch 267
Validation binary_cross_entropy = 0.964099
Epoch 268
Validation binary_cross_entropy = 0.970218
Epoch 269
Loss = 6.6356e-02, PNorm = 67.8870, GNorm = 2.9820, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 1.001966
Epoch 270
Validation binary_cross_entropy = 0.949548
Epoch 271
Validation binary_cross_entropy = 0.882026
Epoch 272
Validation binary_cross_entropy = 0.850530
Epoch 273
Validation binary_cross_entropy = 0.849377
Epoch 274
Loss = 2.1530e-02, PNorm = 67.9460, GNorm = 0.7333, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 0.859311
Epoch 275
Validation binary_cross_entropy = 0.897594
Epoch 276
Validation binary_cross_entropy = 0.988357
Epoch 277
Validation binary_cross_entropy = 0.980737
Epoch 278
Validation binary_cross_entropy = 0.999562
Epoch 279
Loss = 1.1250e-02, PNorm = 67.9977, GNorm = 0.2630, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 1.050128
Epoch 280
Validation binary_cross_entropy = 1.132589
Epoch 281
Validation binary_cross_entropy = 1.350404
Epoch 282
Validation binary_cross_entropy = 1.076796
Epoch 283
Validation binary_cross_entropy = 0.924323
Epoch 284
Loss = 2.7630e-02, PNorm = 68.0418, GNorm = 1.7022, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 0.874866
Epoch 285
Validation binary_cross_entropy = 0.879988
Epoch 286
Validation binary_cross_entropy = 0.899492
Epoch 287
Validation binary_cross_entropy = 0.888202
Epoch 288
Validation binary_cross_entropy = 0.901789
Epoch 289
Loss = 8.6714e-02, PNorm = 68.0950, GNorm = 2.2805, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 0.947751
Epoch 290
Validation binary_cross_entropy = 0.997188
Epoch 291
Validation binary_cross_entropy = 1.072040
Epoch 292
Validation binary_cross_entropy = 1.155267
Epoch 293
Validation binary_cross_entropy = 1.246546
Epoch 294
Loss = 8.2947e-02, PNorm = 68.1399, GNorm = 0.4094, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 1.090367
Epoch 295
Validation binary_cross_entropy = 1.105184
Epoch 296
Validation binary_cross_entropy = 1.107655
Epoch 297
Validation binary_cross_entropy = 1.196965
Epoch 298
Validation binary_cross_entropy = 1.177516
Epoch 299
Loss = 1.1430e-01, PNorm = 68.1812, GNorm = 1.5346, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 1.154961
Model 0 best validation binary_cross_entropy = 0.287189 on epoch 122
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.193636
Ensemble test binary_cross_entropy = 0.193636
Fold 5
Splitting data with seed 5
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.4, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=800, bias=False)
        (W_h): Linear(in_features=800, out_features=800, bias=False)
        (W_o): Linear(in_features=933, out_features=800, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.4, inplace=False)
    (1): Linear(in_features=3100, out_features=800, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.4, inplace=False)
    (4): Linear(in_features=800, out_features=800, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.4, inplace=False)
    (7): Linear(in_features=800, out_features=1, bias=True)
  )
)
Number of parameters = 4,627,201
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 0.289534
Epoch 1
Validation binary_cross_entropy = 0.303106
Epoch 2
Validation binary_cross_entropy = 0.591807
Epoch 3
Validation binary_cross_entropy = 0.330373
Epoch 4
Loss = 7.9908e-01, PNorm = 63.1519, GNorm = 7.7105, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 0.296419
Epoch 5
Validation binary_cross_entropy = 0.581647
Epoch 6
Validation binary_cross_entropy = 0.814832
Epoch 7
Validation binary_cross_entropy = 0.313778
Epoch 8
Validation binary_cross_entropy = 0.428733
Epoch 9
Loss = 7.0348e-01, PNorm = 63.2918, GNorm = 6.8510, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 0.757956
Epoch 10
Validation binary_cross_entropy = 0.331210
Epoch 11
Validation binary_cross_entropy = 0.318045
Epoch 12
Validation binary_cross_entropy = 0.867532
Epoch 13
Validation binary_cross_entropy = 0.479021
Epoch 14
Loss = 8.2486e-01, PNorm = 63.4349, GNorm = 7.0120, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 0.308802
Epoch 15
Validation binary_cross_entropy = 0.827013
Epoch 16
Validation binary_cross_entropy = 0.475661
Epoch 17
Validation binary_cross_entropy = 0.339257
Epoch 18
Validation binary_cross_entropy = 0.470448
Epoch 19
Loss = 6.7259e-01, PNorm = 63.5626, GNorm = 3.9253, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 0.482667
Epoch 20
Validation binary_cross_entropy = 0.373375
Epoch 21
Validation binary_cross_entropy = 0.426444
Epoch 22
Validation binary_cross_entropy = 0.367177
Epoch 23
Validation binary_cross_entropy = 0.346998
Epoch 24
Loss = 6.9311e-01, PNorm = 63.6544, GNorm = 3.2835, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 0.599024
Epoch 25
Validation binary_cross_entropy = 0.702147
Epoch 26
Validation binary_cross_entropy = 0.486602
Epoch 27
Validation binary_cross_entropy = 0.420818
Epoch 28
Validation binary_cross_entropy = 0.463863
Epoch 29
Loss = 7.3363e-01, PNorm = 63.7302, GNorm = 4.3377, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 0.629639
Epoch 30
Validation binary_cross_entropy = 0.407996
Epoch 31
Validation binary_cross_entropy = 0.377826
Epoch 32
Validation binary_cross_entropy = 0.461973
Epoch 33
Validation binary_cross_entropy = 0.694770
Epoch 34
Loss = 6.0167e-01, PNorm = 63.7929, GNorm = 3.0931, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 0.718450
Epoch 35
Validation binary_cross_entropy = 0.418568
Epoch 36
Validation binary_cross_entropy = 0.372510
Epoch 37
Validation binary_cross_entropy = 0.517316
Epoch 38
Validation binary_cross_entropy = 0.573922
Epoch 39
Loss = 6.2582e-01, PNorm = 63.8460, GNorm = 2.6273, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 0.506180
Epoch 40
Validation binary_cross_entropy = 0.449721
Epoch 41
Validation binary_cross_entropy = 0.456236
Epoch 42
Validation binary_cross_entropy = 0.528537
Epoch 43
Validation binary_cross_entropy = 0.576312
Epoch 44
Loss = 6.4338e-01, PNorm = 63.9045, GNorm = 2.7616, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 0.442549
Epoch 45
Validation binary_cross_entropy = 0.399084
Epoch 46
Validation binary_cross_entropy = 0.449642
Epoch 47
Validation binary_cross_entropy = 0.616498
Epoch 48
Validation binary_cross_entropy = 0.657342
Epoch 49
Loss = 5.8795e-01, PNorm = 63.9608, GNorm = 2.2239, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 0.526729
Epoch 50
Validation binary_cross_entropy = 0.453491
Epoch 51
Validation binary_cross_entropy = 0.410425
Epoch 52
Validation binary_cross_entropy = 0.480687
Epoch 53
Validation binary_cross_entropy = 0.599847
Epoch 54
Loss = 5.9766e-01, PNorm = 64.0206, GNorm = 2.2348, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 0.512638
Epoch 55
Validation binary_cross_entropy = 0.367258
Epoch 56
Validation binary_cross_entropy = 0.442367
Epoch 57
Validation binary_cross_entropy = 0.508343
Epoch 58
Validation binary_cross_entropy = 0.389574
Epoch 59
Loss = 4.5220e-01, PNorm = 64.0929, GNorm = 1.9649, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 0.609938
Epoch 60
Validation binary_cross_entropy = 0.388002
Epoch 61
Validation binary_cross_entropy = 0.379866
Epoch 62
Validation binary_cross_entropy = 0.372816
Epoch 63
Validation binary_cross_entropy = 0.473468
Epoch 64
Loss = 5.3463e-01, PNorm = 64.1623, GNorm = 2.0874, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 0.537794
Epoch 65
Validation binary_cross_entropy = 0.342271
Epoch 66
Validation binary_cross_entropy = 0.504715
Epoch 67
Validation binary_cross_entropy = 0.570594
Epoch 68
Validation binary_cross_entropy = 0.305778
Epoch 69
Loss = 5.3503e-01, PNorm = 64.2516, GNorm = 3.6953, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 0.524626
Epoch 70
Validation binary_cross_entropy = 0.718144
Epoch 71
Validation binary_cross_entropy = 0.403082
Epoch 72
Validation binary_cross_entropy = 0.344761
Epoch 73
Validation binary_cross_entropy = 0.483254
Epoch 74
Loss = 6.1048e-01, PNorm = 64.3392, GNorm = 2.8279, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 0.719330
Epoch 75
Validation binary_cross_entropy = 0.615702
Epoch 76
Validation binary_cross_entropy = 0.365460
Epoch 77
Validation binary_cross_entropy = 0.326033
Epoch 78
Validation binary_cross_entropy = 0.632319
Epoch 79
Loss = 5.5140e-01, PNorm = 64.4419, GNorm = 4.0891, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 0.348579
Epoch 80
Validation binary_cross_entropy = 0.317703
Epoch 81
Validation binary_cross_entropy = 0.589992
Epoch 82
Validation binary_cross_entropy = 0.494011
Epoch 83
Validation binary_cross_entropy = 0.358935
Epoch 84
Loss = 5.2401e-01, PNorm = 64.5326, GNorm = 2.0422, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 0.335201
Epoch 85
Validation binary_cross_entropy = 0.353149
Epoch 86
Validation binary_cross_entropy = 0.439620
Epoch 87
Validation binary_cross_entropy = 0.475514
Epoch 88
Validation binary_cross_entropy = 0.340624
Epoch 89
Loss = 4.3803e-01, PNorm = 64.6462, GNorm = 1.6727, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 0.349045
Epoch 90
Validation binary_cross_entropy = 0.380066
Epoch 91
Validation binary_cross_entropy = 0.366312
Epoch 92
Validation binary_cross_entropy = 0.343032
Epoch 93
Validation binary_cross_entropy = 0.418403
Epoch 94
Loss = 4.7908e-01, PNorm = 64.7488, GNorm = 2.7550, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 0.304596
Epoch 95
Validation binary_cross_entropy = 0.359888
Epoch 96
Validation binary_cross_entropy = 0.406973
Epoch 97
Validation binary_cross_entropy = 0.469343
Epoch 98
Validation binary_cross_entropy = 0.305717
Epoch 99
Loss = 4.1188e-01, PNorm = 64.8650, GNorm = 3.1658, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.373944
Epoch 100
Validation binary_cross_entropy = 0.339074
Epoch 101
Validation binary_cross_entropy = 0.419456
Epoch 102
Validation binary_cross_entropy = 0.391161
Epoch 103
Validation binary_cross_entropy = 0.444045
Epoch 104
Loss = 4.3906e-01, PNorm = 64.9807, GNorm = 2.7561, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.333910
Epoch 105
Validation binary_cross_entropy = 0.528069
Epoch 106
Validation binary_cross_entropy = 0.380991
Epoch 107
Validation binary_cross_entropy = 0.283916
Epoch 108
Validation binary_cross_entropy = 0.322393
Epoch 109
Loss = 3.7935e-01, PNorm = 65.0981, GNorm = 1.8180, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.461531
Epoch 110
Validation binary_cross_entropy = 0.351225
Epoch 111
Validation binary_cross_entropy = 0.306402
Epoch 112
Validation binary_cross_entropy = 0.385776
Epoch 113
Validation binary_cross_entropy = 0.452676
Epoch 114
Loss = 3.3111e-01, PNorm = 65.2166, GNorm = 1.5309, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.330124
Epoch 115
Validation binary_cross_entropy = 0.363563
Epoch 116
Validation binary_cross_entropy = 0.395418
Epoch 117
Validation binary_cross_entropy = 0.396988
Epoch 118
Validation binary_cross_entropy = 0.398406
Epoch 119
Loss = 2.2646e-01, PNorm = 65.3474, GNorm = 1.9135, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.725368
Epoch 120
Validation binary_cross_entropy = 0.425772
Epoch 121
Validation binary_cross_entropy = 0.395078
Epoch 122
Validation binary_cross_entropy = 0.446521
Epoch 123
Validation binary_cross_entropy = 0.327633
Epoch 124
Loss = 3.6488e-01, PNorm = 65.4679, GNorm = 2.2302, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.322538
Epoch 125
Validation binary_cross_entropy = 0.393764
Epoch 126
Validation binary_cross_entropy = 0.426126
Epoch 127
Validation binary_cross_entropy = 0.393221
Epoch 128
Validation binary_cross_entropy = 0.438141
Epoch 129
Loss = 2.6660e-01, PNorm = 65.6184, GNorm = 2.3785, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.558798
Epoch 130
Validation binary_cross_entropy = 0.497348
Epoch 131
Validation binary_cross_entropy = 0.737775
Epoch 132
Validation binary_cross_entropy = 0.519649
Epoch 133
Validation binary_cross_entropy = 0.451083
Epoch 134
Loss = 2.3550e-01, PNorm = 65.7456, GNorm = 1.1615, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.483131
Epoch 135
Validation binary_cross_entropy = 0.556122
Epoch 136
Validation binary_cross_entropy = 0.431436
Epoch 137
Validation binary_cross_entropy = 0.420294
Epoch 138
Validation binary_cross_entropy = 0.467153
Epoch 139
Loss = 1.8987e-01, PNorm = 65.8634, GNorm = 1.2742, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.565086
Epoch 140
Validation binary_cross_entropy = 0.539815
Epoch 141
Validation binary_cross_entropy = 0.558009
Epoch 142
Validation binary_cross_entropy = 0.595306
Epoch 143
Validation binary_cross_entropy = 0.583532
Epoch 144
Loss = 2.5914e-01, PNorm = 65.9621, GNorm = 1.3477, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.554689
Epoch 145
Validation binary_cross_entropy = 0.535855
Epoch 146
Validation binary_cross_entropy = 0.530141
Epoch 147
Validation binary_cross_entropy = 0.516437
Epoch 148
Validation binary_cross_entropy = 0.532888
Epoch 149
Loss = 2.7930e-01, PNorm = 66.0599, GNorm = 2.5731, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.547819
Epoch 150
Validation binary_cross_entropy = 0.623586
Epoch 151
Validation binary_cross_entropy = 0.617215
Epoch 152
Validation binary_cross_entropy = 0.631802
Epoch 153
Validation binary_cross_entropy = 0.614046
Epoch 154
Loss = 1.8617e-01, PNorm = 66.1565, GNorm = 1.0143, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.614184
Epoch 155
Validation binary_cross_entropy = 0.582074
Epoch 156
Validation binary_cross_entropy = 0.636387
Epoch 157
Validation binary_cross_entropy = 0.591666
Epoch 158
Validation binary_cross_entropy = 0.612196
Epoch 159
Loss = 1.4365e-01, PNorm = 66.2581, GNorm = 2.6039, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.593060
Epoch 160
Validation binary_cross_entropy = 0.617739
Epoch 161
Validation binary_cross_entropy = 0.645798
Epoch 162
Validation binary_cross_entropy = 0.712961
Epoch 163
Validation binary_cross_entropy = 0.653916
Epoch 164
Loss = 1.4867e-01, PNorm = 66.3599, GNorm = 2.2036, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.673173
Epoch 165
Validation binary_cross_entropy = 0.686101
Epoch 166
Validation binary_cross_entropy = 0.636391
Epoch 167
Validation binary_cross_entropy = 0.605493
Epoch 168
Validation binary_cross_entropy = 0.611337
Epoch 169
Loss = 1.2600e-01, PNorm = 66.4578, GNorm = 0.7406, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.634608
Epoch 170
Validation binary_cross_entropy = 0.656961
Epoch 171
Validation binary_cross_entropy = 0.705853
Epoch 172
Validation binary_cross_entropy = 0.740981
Epoch 173
Validation binary_cross_entropy = 0.811262
Epoch 174
Loss = 1.8654e-01, PNorm = 66.5507, GNorm = 0.7556, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 0.690891
Epoch 175
Validation binary_cross_entropy = 0.662704
Epoch 176
Validation binary_cross_entropy = 0.685672
Epoch 177
Validation binary_cross_entropy = 0.692484
Epoch 178
Validation binary_cross_entropy = 0.660337
Epoch 179
Loss = 1.0502e-01, PNorm = 66.6497, GNorm = 1.1772, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 0.752122
Epoch 180
Validation binary_cross_entropy = 0.722603
Epoch 181
Validation binary_cross_entropy = 0.768387
Epoch 182
Validation binary_cross_entropy = 0.741738
Epoch 183
Validation binary_cross_entropy = 0.625711
Epoch 184
Loss = 2.2870e-01, PNorm = 66.7444, GNorm = 3.7996, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 0.590199
Epoch 185
Validation binary_cross_entropy = 0.537927
Epoch 186
Validation binary_cross_entropy = 0.553995
Epoch 187
Validation binary_cross_entropy = 0.579386
Epoch 188
Validation binary_cross_entropy = 0.607008
Epoch 189
Loss = 1.0560e-01, PNorm = 66.8375, GNorm = 0.8903, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 0.666027
Epoch 190
Validation binary_cross_entropy = 0.688308
Epoch 191
Validation binary_cross_entropy = 0.703030
Epoch 192
Validation binary_cross_entropy = 0.736917
Epoch 193
Validation binary_cross_entropy = 0.739970
Epoch 194
Loss = 1.3045e-01, PNorm = 66.9184, GNorm = 2.0950, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 0.750743
Epoch 195
Validation binary_cross_entropy = 0.793034
Epoch 196
Validation binary_cross_entropy = 0.852751
Epoch 197
Validation binary_cross_entropy = 0.874709
Epoch 198
Validation binary_cross_entropy = 0.913841
Epoch 199
Loss = 9.6365e-02, PNorm = 66.9964, GNorm = 0.9199, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 0.928183
Epoch 200
Validation binary_cross_entropy = 0.964770
Epoch 201
Validation binary_cross_entropy = 0.992088
Epoch 202
Validation binary_cross_entropy = 0.906736
Epoch 203
Validation binary_cross_entropy = 0.840484
Epoch 204
Loss = 1.5256e-01, PNorm = 67.0575, GNorm = 1.4163, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 0.799154
Epoch 205
Validation binary_cross_entropy = 0.770710
Epoch 206
Validation binary_cross_entropy = 0.694112
Epoch 207
Validation binary_cross_entropy = 0.690733
Epoch 208
Validation binary_cross_entropy = 0.684760
Epoch 209
Loss = 1.5869e-01, PNorm = 67.1452, GNorm = 1.2159, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 0.689744
Epoch 210
Validation binary_cross_entropy = 0.743307
Epoch 211
Validation binary_cross_entropy = 0.740692
Epoch 212
Validation binary_cross_entropy = 0.710989
Epoch 213
Validation binary_cross_entropy = 0.729292
Epoch 214
Loss = 9.0190e-02, PNorm = 67.2293, GNorm = 1.6501, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 0.773046
Epoch 215
Validation binary_cross_entropy = 0.850831
Epoch 216
Validation binary_cross_entropy = 0.977448
Epoch 217
Validation binary_cross_entropy = 1.042622
Epoch 218
Validation binary_cross_entropy = 0.915854
Epoch 219
Loss = 1.1593e-01, PNorm = 67.3031, GNorm = 1.5969, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 0.899561
Epoch 220
Validation binary_cross_entropy = 0.933875
Epoch 221
Validation binary_cross_entropy = 1.011824
Epoch 222
Validation binary_cross_entropy = 0.887529
Epoch 223
Validation binary_cross_entropy = 0.798345
Epoch 224
Loss = 2.6421e-01, PNorm = 67.3826, GNorm = 3.1759, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 0.718095
Epoch 225
Validation binary_cross_entropy = 0.626909
Epoch 226
Validation binary_cross_entropy = 0.623843
Epoch 227
Validation binary_cross_entropy = 0.628944
Epoch 228
Validation binary_cross_entropy = 0.638889
Epoch 229
Loss = 1.1235e-01, PNorm = 67.4683, GNorm = 0.9846, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 0.666587
Epoch 230
Validation binary_cross_entropy = 0.720508
Epoch 231
Validation binary_cross_entropy = 0.802497
Epoch 232
Validation binary_cross_entropy = 0.925541
Epoch 233
Validation binary_cross_entropy = 1.054983
Epoch 234
Loss = 1.2048e-01, PNorm = 67.5443, GNorm = 2.8026, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 0.991731
Epoch 235
Validation binary_cross_entropy = 0.983692
Epoch 236
Validation binary_cross_entropy = 1.019579
Epoch 237
Validation binary_cross_entropy = 1.046572
Epoch 238
Validation binary_cross_entropy = 1.068115
Epoch 239
Loss = 1.0555e-01, PNorm = 67.6055, GNorm = 0.5131, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 1.073397
Epoch 240
Validation binary_cross_entropy = 1.095601
Epoch 241
Validation binary_cross_entropy = 1.019189
Epoch 242
Validation binary_cross_entropy = 0.952670
Epoch 243
Validation binary_cross_entropy = 0.917772
Epoch 244
Loss = 1.6130e-01, PNorm = 67.6691, GNorm = 3.0270, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 0.896089
Epoch 245
Validation binary_cross_entropy = 0.965748
Epoch 246
Validation binary_cross_entropy = 0.921935
Epoch 247
Validation binary_cross_entropy = 0.864383
Epoch 248
Validation binary_cross_entropy = 0.861341
Epoch 249
Loss = 1.2510e-01, PNorm = 67.7294, GNorm = 1.6486, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 0.909285
Epoch 250
Validation binary_cross_entropy = 0.975559
Epoch 251
Validation binary_cross_entropy = 1.029051
Epoch 252
Validation binary_cross_entropy = 1.051942
Epoch 253
Validation binary_cross_entropy = 1.028209
Epoch 254
Loss = 4.6535e-02, PNorm = 67.7896, GNorm = 0.8437, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 1.044429
Epoch 255
Validation binary_cross_entropy = 1.108052
Epoch 256
Validation binary_cross_entropy = 1.217584
Epoch 257
Validation binary_cross_entropy = 1.258634
Epoch 258
Validation binary_cross_entropy = 1.113582
Epoch 259
Loss = 1.9162e-01, PNorm = 67.8421, GNorm = 1.9296, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 0.957873
Epoch 260
Validation binary_cross_entropy = 0.875512
Epoch 261
Validation binary_cross_entropy = 0.863050
Epoch 262
Validation binary_cross_entropy = 0.868029
Epoch 263
Validation binary_cross_entropy = 0.853969
Epoch 264
Loss = 9.6258e-02, PNorm = 67.9025, GNorm = 0.8895, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 0.842791
Epoch 265
Validation binary_cross_entropy = 0.853808
Epoch 266
Validation binary_cross_entropy = 0.926803
Epoch 267
Validation binary_cross_entropy = 1.036205
Epoch 268
Validation binary_cross_entropy = 1.101811
Epoch 269
Loss = 4.7476e-02, PNorm = 67.9559, GNorm = 1.1950, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 1.068694
Epoch 270
Validation binary_cross_entropy = 1.070621
Epoch 271
Validation binary_cross_entropy = 1.080504
Epoch 272
Validation binary_cross_entropy = 1.150066
Epoch 273
Validation binary_cross_entropy = 1.288207
Epoch 274
Loss = 9.9065e-02, PNorm = 68.0014, GNorm = 5.5152, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 1.286754
Epoch 275
Validation binary_cross_entropy = 1.195713
Epoch 276
Validation binary_cross_entropy = 1.163258
Epoch 277
Validation binary_cross_entropy = 1.175258
Epoch 278
Validation binary_cross_entropy = 1.191755
Epoch 279
Loss = 8.4779e-02, PNorm = 68.0459, GNorm = 1.1401, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 1.257287
Epoch 280
Validation binary_cross_entropy = 1.257581
Epoch 281
Validation binary_cross_entropy = 1.193638
Epoch 282
Validation binary_cross_entropy = 1.171147
Epoch 283
Validation binary_cross_entropy = 1.180408
Epoch 284
Loss = 1.3435e-01, PNorm = 68.0841, GNorm = 1.9187, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 1.184433
Epoch 285
Validation binary_cross_entropy = 1.217888
Epoch 286
Validation binary_cross_entropy = 1.231984
Epoch 287
Validation binary_cross_entropy = 1.113955
Epoch 288
Validation binary_cross_entropy = 1.041147
Epoch 289
Loss = 5.8976e-02, PNorm = 68.1353, GNorm = 0.8985, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 1.037846
Epoch 290
Validation binary_cross_entropy = 1.024491
Epoch 291
Validation binary_cross_entropy = 1.065444
Epoch 292
Validation binary_cross_entropy = 1.081721
Epoch 293
Validation binary_cross_entropy = 1.018857
Epoch 294
Loss = 4.7834e-02, PNorm = 68.1844, GNorm = 0.4974, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 0.998081
Epoch 295
Validation binary_cross_entropy = 0.990471
Epoch 296
Validation binary_cross_entropy = 1.017386
Epoch 297
Validation binary_cross_entropy = 1.089213
Epoch 298
Validation binary_cross_entropy = 1.123882
Epoch 299
Loss = 4.1473e-02, PNorm = 68.2332, GNorm = 0.9392, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 1.135837
Model 0 best validation binary_cross_entropy = 0.283916 on epoch 107
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.211122
Ensemble test binary_cross_entropy = 0.211122
Fold 6
Splitting data with seed 6
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.4, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=800, bias=False)
        (W_h): Linear(in_features=800, out_features=800, bias=False)
        (W_o): Linear(in_features=933, out_features=800, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.4, inplace=False)
    (1): Linear(in_features=3100, out_features=800, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.4, inplace=False)
    (4): Linear(in_features=800, out_features=800, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.4, inplace=False)
    (7): Linear(in_features=800, out_features=1, bias=True)
  )
)
Number of parameters = 4,627,201
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 0.293489
Epoch 1
Validation binary_cross_entropy = 0.301747
Epoch 2
Validation binary_cross_entropy = 0.614025
Epoch 3
Validation binary_cross_entropy = 0.314358
Epoch 4
Loss = 9.0234e-01, PNorm = 63.1523, GNorm = 8.5275, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 0.292968
Epoch 5
Validation binary_cross_entropy = 0.569553
Epoch 6
Validation binary_cross_entropy = 0.701773
Epoch 7
Validation binary_cross_entropy = 0.312632
Epoch 8
Validation binary_cross_entropy = 0.472939
Epoch 9
Loss = 7.2079e-01, PNorm = 63.2846, GNorm = 5.3228, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 0.644445
Epoch 10
Validation binary_cross_entropy = 0.339892
Epoch 11
Validation binary_cross_entropy = 0.354199
Epoch 12
Validation binary_cross_entropy = 0.683794
Epoch 13
Validation binary_cross_entropy = 0.325539
Epoch 14
Loss = 7.7695e-01, PNorm = 63.4385, GNorm = 5.8793, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 0.317181
Epoch 15
Validation binary_cross_entropy = 1.107939
Epoch 16
Validation binary_cross_entropy = 0.365324
Epoch 17
Validation binary_cross_entropy = 0.296562
Epoch 18
Validation binary_cross_entropy = 0.521165
Epoch 19
Loss = 7.6414e-01, PNorm = 63.5639, GNorm = 6.9624, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 1.012209
Epoch 20
Validation binary_cross_entropy = 0.359591
Epoch 21
Validation binary_cross_entropy = 0.302680
Epoch 22
Validation binary_cross_entropy = 0.396236
Epoch 23
Validation binary_cross_entropy = 0.675818
Epoch 24
Loss = 6.7859e-01, PNorm = 63.6587, GNorm = 4.6212, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 0.603887
Epoch 25
Validation binary_cross_entropy = 0.344314
Epoch 26
Validation binary_cross_entropy = 0.344532
Epoch 27
Validation binary_cross_entropy = 0.449716
Epoch 28
Validation binary_cross_entropy = 0.545673
Epoch 29
Loss = 6.6723e-01, PNorm = 63.7292, GNorm = 2.7616, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 0.454104
Epoch 30
Validation binary_cross_entropy = 0.403414
Epoch 31
Validation binary_cross_entropy = 0.404294
Epoch 32
Validation binary_cross_entropy = 0.496384
Epoch 33
Validation binary_cross_entropy = 0.497117
Epoch 34
Loss = 5.5663e-01, PNorm = 63.7932, GNorm = 2.4544, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 0.546613
Epoch 35
Validation binary_cross_entropy = 0.527903
Epoch 36
Validation binary_cross_entropy = 0.482495
Epoch 37
Validation binary_cross_entropy = 0.515088
Epoch 38
Validation binary_cross_entropy = 0.480556
Epoch 39
Loss = 6.6443e-01, PNorm = 63.8496, GNorm = 3.0463, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 0.417855
Epoch 40
Validation binary_cross_entropy = 0.428011
Epoch 41
Validation binary_cross_entropy = 0.596220
Epoch 42
Validation binary_cross_entropy = 0.606077
Epoch 43
Validation binary_cross_entropy = 0.438176
Epoch 44
Loss = 5.6428e-01, PNorm = 63.9044, GNorm = 3.1183, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 0.406244
Epoch 45
Validation binary_cross_entropy = 0.545882
Epoch 46
Validation binary_cross_entropy = 0.577568
Epoch 47
Validation binary_cross_entropy = 0.458864
Epoch 48
Validation binary_cross_entropy = 0.465031
Epoch 49
Loss = 5.6304e-01, PNorm = 63.9610, GNorm = 2.1331, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 0.493329
Epoch 50
Validation binary_cross_entropy = 0.531007
Epoch 51
Validation binary_cross_entropy = 0.533820
Epoch 52
Validation binary_cross_entropy = 0.491103
Epoch 53
Validation binary_cross_entropy = 0.419268
Epoch 54
Loss = 5.3976e-01, PNorm = 64.0153, GNorm = 2.2686, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 0.440039
Epoch 55
Validation binary_cross_entropy = 0.537297
Epoch 56
Validation binary_cross_entropy = 0.549532
Epoch 57
Validation binary_cross_entropy = 0.469908
Epoch 58
Validation binary_cross_entropy = 0.411494
Epoch 59
Loss = 6.4141e-01, PNorm = 64.0731, GNorm = 2.3230, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 0.447328
Epoch 60
Validation binary_cross_entropy = 0.597291
Epoch 61
Validation binary_cross_entropy = 0.660272
Epoch 62
Validation binary_cross_entropy = 0.548013
Epoch 63
Validation binary_cross_entropy = 0.441398
Epoch 64
Loss = 6.0195e-01, PNorm = 64.1460, GNorm = 2.3538, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 0.392233
Epoch 65
Validation binary_cross_entropy = 0.438474
Epoch 66
Validation binary_cross_entropy = 0.580010
Epoch 67
Validation binary_cross_entropy = 0.697825
Epoch 68
Validation binary_cross_entropy = 0.525521
Epoch 69
Loss = 5.3425e-01, PNorm = 64.2159, GNorm = 2.1359, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 0.393212
Epoch 70
Validation binary_cross_entropy = 0.409103
Epoch 71
Validation binary_cross_entropy = 0.491681
Epoch 72
Validation binary_cross_entropy = 0.414319
Epoch 73
Validation binary_cross_entropy = 0.476861
Epoch 74
Loss = 5.8340e-01, PNorm = 64.2903, GNorm = 2.1058, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 0.682040
Epoch 75
Validation binary_cross_entropy = 0.647853
Epoch 76
Validation binary_cross_entropy = 0.420309
Epoch 77
Validation binary_cross_entropy = 0.343876
Epoch 78
Validation binary_cross_entropy = 0.610494
Epoch 79
Loss = 6.4652e-01, PNorm = 64.3904, GNorm = 3.0685, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 0.288250
Epoch 80
Validation binary_cross_entropy = 0.387944
Epoch 81
Validation binary_cross_entropy = 0.597684
Epoch 82
Validation binary_cross_entropy = 0.712290
Epoch 83
Validation binary_cross_entropy = 0.592159
Epoch 84
Loss = 5.2166e-01, PNorm = 64.4952, GNorm = 1.4487, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 0.446120
Epoch 85
Validation binary_cross_entropy = 0.394136
Epoch 86
Validation binary_cross_entropy = 0.471795
Epoch 87
Validation binary_cross_entropy = 0.539885
Epoch 88
Validation binary_cross_entropy = 0.479185
Epoch 89
Loss = 4.4481e-01, PNorm = 64.6165, GNorm = 1.6449, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 0.408270
Epoch 90
Validation binary_cross_entropy = 0.425316
Epoch 91
Validation binary_cross_entropy = 0.495722
Epoch 92
Validation binary_cross_entropy = 0.386988
Epoch 93
Validation binary_cross_entropy = 0.410314
Epoch 94
Loss = 3.9783e-01, PNorm = 64.7317, GNorm = 2.5933, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 0.363628
Epoch 95
Validation binary_cross_entropy = 0.378598
Epoch 96
Validation binary_cross_entropy = 0.472420
Epoch 97
Validation binary_cross_entropy = 0.365591
Epoch 98
Validation binary_cross_entropy = 0.430086
Epoch 99
Loss = 3.8539e-01, PNorm = 64.8403, GNorm = 1.8466, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.459187
Epoch 100
Validation binary_cross_entropy = 0.399705
Epoch 101
Validation binary_cross_entropy = 0.397056
Epoch 102
Validation binary_cross_entropy = 0.556238
Epoch 103
Validation binary_cross_entropy = 0.377706
Epoch 104
Loss = 3.2994e-01, PNorm = 64.9784, GNorm = 2.0063, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.589278
Epoch 105
Validation binary_cross_entropy = 0.305979
Epoch 106
Validation binary_cross_entropy = 0.376418
Epoch 107
Validation binary_cross_entropy = 0.469289
Epoch 108
Validation binary_cross_entropy = 0.337849
Epoch 109
Loss = 3.4071e-01, PNorm = 65.1220, GNorm = 1.5729, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.324955
Epoch 110
Validation binary_cross_entropy = 0.465295
Epoch 111
Validation binary_cross_entropy = 0.335518
Epoch 112
Validation binary_cross_entropy = 0.303083
Epoch 113
Validation binary_cross_entropy = 0.413269
Epoch 114
Loss = 3.2908e-01, PNorm = 65.2556, GNorm = 1.8398, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.335452
Epoch 115
Validation binary_cross_entropy = 0.301535
Epoch 116
Validation binary_cross_entropy = 0.520122
Epoch 117
Validation binary_cross_entropy = 0.366098
Epoch 118
Validation binary_cross_entropy = 0.312534
Epoch 119
Loss = 3.1951e-01, PNorm = 65.4008, GNorm = 2.6826, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.387874
Epoch 120
Validation binary_cross_entropy = 0.470964
Epoch 121
Validation binary_cross_entropy = 0.385001
Epoch 122
Validation binary_cross_entropy = 0.404180
Epoch 123
Validation binary_cross_entropy = 0.415754
Epoch 124
Loss = 2.8206e-01, PNorm = 65.5385, GNorm = 1.8658, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.418048
Epoch 125
Validation binary_cross_entropy = 0.387753
Epoch 126
Validation binary_cross_entropy = 0.417011
Epoch 127
Validation binary_cross_entropy = 0.365134
Epoch 128
Validation binary_cross_entropy = 0.374518
Epoch 129
Loss = 2.4043e-01, PNorm = 65.6751, GNorm = 1.8044, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.420426
Epoch 130
Validation binary_cross_entropy = 0.451899
Epoch 131
Validation binary_cross_entropy = 0.512231
Epoch 132
Validation binary_cross_entropy = 0.508906
Epoch 133
Validation binary_cross_entropy = 0.396655
Epoch 134
Loss = 2.0624e-01, PNorm = 65.7930, GNorm = 3.7620, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.368230
Epoch 135
Validation binary_cross_entropy = 0.370034
Epoch 136
Validation binary_cross_entropy = 0.375058
Epoch 137
Validation binary_cross_entropy = 0.389770
Epoch 138
Validation binary_cross_entropy = 0.430091
Epoch 139
Loss = 2.7396e-01, PNorm = 65.9172, GNorm = 3.2256, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.472394
Epoch 140
Validation binary_cross_entropy = 0.529886
Epoch 141
Validation binary_cross_entropy = 0.490586
Epoch 142
Validation binary_cross_entropy = 0.475437
Epoch 143
Validation binary_cross_entropy = 0.475941
Epoch 144
Loss = 2.0885e-01, PNorm = 66.0169, GNorm = 1.4926, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.433985
Epoch 145
Validation binary_cross_entropy = 0.439605
Epoch 146
Validation binary_cross_entropy = 0.454332
Epoch 147
Validation binary_cross_entropy = 0.497678
Epoch 148
Validation binary_cross_entropy = 0.543095
Epoch 149
Loss = 1.8164e-01, PNorm = 66.1313, GNorm = 2.3805, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.623861
Epoch 150
Validation binary_cross_entropy = 0.595238
Epoch 151
Validation binary_cross_entropy = 0.583996
Epoch 152
Validation binary_cross_entropy = 0.694042
Epoch 153
Validation binary_cross_entropy = 0.432960
Epoch 154
Loss = 3.0347e-01, PNorm = 66.2375, GNorm = 4.2439, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.425166
Epoch 155
Validation binary_cross_entropy = 0.375888
Epoch 156
Validation binary_cross_entropy = 0.420325
Epoch 157
Validation binary_cross_entropy = 0.466099
Epoch 158
Validation binary_cross_entropy = 0.536407
Epoch 159
Loss = 2.7980e-01, PNorm = 66.3818, GNorm = 1.2861, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.493207
Epoch 160
Validation binary_cross_entropy = 0.453295
Epoch 161
Validation binary_cross_entropy = 0.398143
Epoch 162
Validation binary_cross_entropy = 0.408703
Epoch 163
Validation binary_cross_entropy = 0.420514
Epoch 164
Loss = 1.6335e-01, PNorm = 66.4884, GNorm = 0.9692, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.457490
Epoch 165
Validation binary_cross_entropy = 0.526412
Epoch 166
Validation binary_cross_entropy = 0.555573
Epoch 167
Validation binary_cross_entropy = 0.598804
Epoch 168
Validation binary_cross_entropy = 0.684543
Epoch 169
Loss = 1.2683e-01, PNorm = 66.5932, GNorm = 2.1735, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.732442
Epoch 170
Validation binary_cross_entropy = 0.769169
Epoch 171
Validation binary_cross_entropy = 0.779376
Epoch 172
Validation binary_cross_entropy = 0.823284
Epoch 173
Validation binary_cross_entropy = 0.650657
Epoch 174
Loss = 1.7297e-01, PNorm = 66.6783, GNorm = 2.7252, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 0.623276
Epoch 175
Validation binary_cross_entropy = 0.513569
Epoch 176
Validation binary_cross_entropy = 0.495311
Epoch 177
Validation binary_cross_entropy = 0.538038
Epoch 178
Validation binary_cross_entropy = 0.600122
Epoch 179
Loss = 2.0669e-01, PNorm = 66.7669, GNorm = 2.9143, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 0.588863
Epoch 180
Validation binary_cross_entropy = 0.547671
Epoch 181
Validation binary_cross_entropy = 0.551308
Epoch 182
Validation binary_cross_entropy = 0.580595
Epoch 183
Validation binary_cross_entropy = 0.653159
Epoch 184
Loss = 1.1277e-01, PNorm = 66.8584, GNorm = 1.9201, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 0.670056
Epoch 185
Validation binary_cross_entropy = 0.690236
Epoch 186
Validation binary_cross_entropy = 0.730153
Epoch 187
Validation binary_cross_entropy = 0.697845
Epoch 188
Validation binary_cross_entropy = 0.709235
Epoch 189
Loss = 1.4351e-01, PNorm = 66.9443, GNorm = 1.6607, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 0.682350
Epoch 190
Validation binary_cross_entropy = 0.683389
Epoch 191
Validation binary_cross_entropy = 0.713247
Epoch 192
Validation binary_cross_entropy = 0.718493
Epoch 193
Validation binary_cross_entropy = 0.732184
Epoch 194
Loss = 9.9132e-02, PNorm = 67.0187, GNorm = 1.3884, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 0.761181
Epoch 195
Validation binary_cross_entropy = 0.778129
Epoch 196
Validation binary_cross_entropy = 0.787114
Epoch 197
Validation binary_cross_entropy = 0.803544
Epoch 198
Validation binary_cross_entropy = 0.781996
Epoch 199
Loss = 7.4685e-02, PNorm = 67.0802, GNorm = 0.5584, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 0.653189
Epoch 200
Validation binary_cross_entropy = 0.634238
Epoch 201
Validation binary_cross_entropy = 0.564254
Epoch 202
Validation binary_cross_entropy = 0.562341
Epoch 203
Validation binary_cross_entropy = 0.580927
Epoch 204
Loss = 1.9451e-01, PNorm = 67.1443, GNorm = 0.5528, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 0.579791
Epoch 205
Validation binary_cross_entropy = 0.606667
Epoch 206
Validation binary_cross_entropy = 0.614933
Epoch 207
Validation binary_cross_entropy = 0.645336
Epoch 208
Validation binary_cross_entropy = 0.714849
Epoch 209
Loss = 9.1322e-02, PNorm = 67.2126, GNorm = 1.1719, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 0.756040
Epoch 210
Validation binary_cross_entropy = 0.703131
Epoch 211
Validation binary_cross_entropy = 0.702410
Epoch 212
Validation binary_cross_entropy = 0.740334
Epoch 213
Validation binary_cross_entropy = 0.795591
Epoch 214
Loss = 1.0825e-01, PNorm = 67.2867, GNorm = 1.2293, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 0.776975
Epoch 215
Validation binary_cross_entropy = 0.746601
Epoch 216
Validation binary_cross_entropy = 0.747185
Epoch 217
Validation binary_cross_entropy = 0.761786
Epoch 218
Validation binary_cross_entropy = 0.779205
Epoch 219
Loss = 4.9489e-02, PNorm = 67.3589, GNorm = 1.6405, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 0.769287
Epoch 220
Validation binary_cross_entropy = 0.754499
Epoch 221
Validation binary_cross_entropy = 0.772852
Epoch 222
Validation binary_cross_entropy = 0.803626
Epoch 223
Validation binary_cross_entropy = 0.815199
Epoch 224
Loss = 6.0981e-02, PNorm = 67.4177, GNorm = 1.4392, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 0.793646
Epoch 225
Validation binary_cross_entropy = 0.771697
Epoch 226
Validation binary_cross_entropy = 0.769642
Epoch 227
Validation binary_cross_entropy = 0.821159
Epoch 228
Validation binary_cross_entropy = 0.826643
Epoch 229
Loss = 2.1846e-02, PNorm = 67.4676, GNorm = 0.4477, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 0.796813
Epoch 230
Validation binary_cross_entropy = 0.786348
Epoch 231
Validation binary_cross_entropy = 0.783144
Epoch 232
Validation binary_cross_entropy = 0.780293
Epoch 233
Validation binary_cross_entropy = 0.777062
Epoch 234
Loss = 1.0163e-01, PNorm = 67.5226, GNorm = 1.9619, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 0.831205
Epoch 235
Validation binary_cross_entropy = 0.811773
Epoch 236
Validation binary_cross_entropy = 0.748357
Epoch 237
Validation binary_cross_entropy = 0.740677
Epoch 238
Validation binary_cross_entropy = 0.759986
Epoch 239
Loss = 4.7129e-02, PNorm = 67.5819, GNorm = 0.6335, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 0.786416
Epoch 240
Validation binary_cross_entropy = 0.821375
Epoch 241
Validation binary_cross_entropy = 0.879624
Epoch 242
Validation binary_cross_entropy = 0.891403
Epoch 243
Validation binary_cross_entropy = 0.874506
Epoch 244
Loss = 4.4674e-02, PNorm = 67.6467, GNorm = 0.3946, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 0.816510
Epoch 245
Validation binary_cross_entropy = 0.785812
Epoch 246
Validation binary_cross_entropy = 0.822601
Epoch 247
Validation binary_cross_entropy = 0.839148
Epoch 248
Validation binary_cross_entropy = 0.797250
Epoch 249
Loss = 3.1088e-02, PNorm = 67.7285, GNorm = 0.6917, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 0.812645
Epoch 250
Validation binary_cross_entropy = 0.836272
Epoch 251
Validation binary_cross_entropy = 0.905774
Epoch 252
Validation binary_cross_entropy = 0.931950
Epoch 253
Validation binary_cross_entropy = 0.793924
Epoch 254
Loss = 1.0904e-01, PNorm = 67.7938, GNorm = 1.6225, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 0.697587
Epoch 255
Validation binary_cross_entropy = 0.680482
Epoch 256
Validation binary_cross_entropy = 0.684938
Epoch 257
Validation binary_cross_entropy = 0.705648
Epoch 258
Validation binary_cross_entropy = 0.705974
Epoch 259
Loss = 5.6002e-02, PNorm = 67.8455, GNorm = 0.9963, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 0.720226
Epoch 260
Validation binary_cross_entropy = 0.748949
Epoch 261
Validation binary_cross_entropy = 0.790919
Epoch 262
Validation binary_cross_entropy = 0.835619
Epoch 263
Validation binary_cross_entropy = 0.862126
Epoch 264
Loss = 4.5151e-02, PNorm = 67.8979, GNorm = 1.4306, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 0.889121
Epoch 265
Validation binary_cross_entropy = 0.909027
Epoch 266
Validation binary_cross_entropy = 0.931612
Epoch 267
Validation binary_cross_entropy = 0.956948
Epoch 268
Validation binary_cross_entropy = 0.984944
Epoch 269
Loss = 4.6895e-02, PNorm = 67.9426, GNorm = 1.6135, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 1.013820
Epoch 270
Validation binary_cross_entropy = 0.963109
Epoch 271
Validation binary_cross_entropy = 0.990398
Epoch 272
Validation binary_cross_entropy = 0.955213
Epoch 273
Validation binary_cross_entropy = 1.027863
Epoch 274
Loss = 9.8111e-02, PNorm = 67.9907, GNorm = 1.4186, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 1.092378
Epoch 275
Validation binary_cross_entropy = 1.052894
Epoch 276
Validation binary_cross_entropy = 0.980557
Epoch 277
Validation binary_cross_entropy = 0.906990
Epoch 278
Validation binary_cross_entropy = 0.875074
Epoch 279
Loss = 1.0945e-01, PNorm = 68.0396, GNorm = 0.1009, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 0.956885
Epoch 280
Validation binary_cross_entropy = 1.068989
Epoch 281
Validation binary_cross_entropy = 0.979120
Epoch 282
Validation binary_cross_entropy = 0.798740
Epoch 283
Validation binary_cross_entropy = 0.739207
Epoch 284
Loss = 8.1055e-02, PNorm = 68.0874, GNorm = 0.9133, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 0.737170
Epoch 285
Validation binary_cross_entropy = 0.751274
Epoch 286
Validation binary_cross_entropy = 0.768261
Epoch 287
Validation binary_cross_entropy = 0.768341
Epoch 288
Validation binary_cross_entropy = 0.765707
Epoch 289
Loss = 3.9521e-02, PNorm = 68.1373, GNorm = 0.3561, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 0.774728
Epoch 290
Validation binary_cross_entropy = 0.796114
Epoch 291
Validation binary_cross_entropy = 0.830233
Epoch 292
Validation binary_cross_entropy = 0.885370
Epoch 293
Validation binary_cross_entropy = 0.929380
Epoch 294
Loss = 4.8610e-02, PNorm = 68.1756, GNorm = 0.9885, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 0.938708
Epoch 295
Validation binary_cross_entropy = 0.916867
Epoch 296
Validation binary_cross_entropy = 0.910384
Epoch 297
Validation binary_cross_entropy = 0.928141
Epoch 298
Validation binary_cross_entropy = 0.966286
Epoch 299
Loss = 4.0095e-02, PNorm = 68.2091, GNorm = 1.5233, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 1.059284
Model 0 best validation binary_cross_entropy = 0.288250 on epoch 79
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.227363
Ensemble test binary_cross_entropy = 0.227363
Fold 7
Splitting data with seed 7
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.4, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=800, bias=False)
        (W_h): Linear(in_features=800, out_features=800, bias=False)
        (W_o): Linear(in_features=933, out_features=800, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.4, inplace=False)
    (1): Linear(in_features=3100, out_features=800, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.4, inplace=False)
    (4): Linear(in_features=800, out_features=800, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.4, inplace=False)
    (7): Linear(in_features=800, out_features=1, bias=True)
  )
)
Number of parameters = 4,627,201
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 0.287578
Epoch 1
Validation binary_cross_entropy = 0.297397
Epoch 2
Validation binary_cross_entropy = 0.590341
Epoch 3
Validation binary_cross_entropy = 0.310910
Epoch 4
Loss = 9.0203e-01, PNorm = 63.1507, GNorm = 5.0038, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 0.299353
Epoch 5
Validation binary_cross_entropy = 0.396616
Epoch 6
Validation binary_cross_entropy = 0.793094
Epoch 7
Validation binary_cross_entropy = 0.293812
Epoch 8
Validation binary_cross_entropy = 0.344589
Epoch 9
Loss = 7.5659e-01, PNorm = 63.2866, GNorm = 5.1372, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 0.988440
Epoch 10
Validation binary_cross_entropy = 0.341280
Epoch 11
Validation binary_cross_entropy = 0.296808
Epoch 12
Validation binary_cross_entropy = 0.587465
Epoch 13
Validation binary_cross_entropy = 0.562724
Epoch 14
Loss = 7.0592e-01, PNorm = 63.4369, GNorm = 4.6468, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 0.358044
Epoch 15
Validation binary_cross_entropy = 0.408181
Epoch 16
Validation binary_cross_entropy = 0.459692
Epoch 17
Validation binary_cross_entropy = 0.547062
Epoch 18
Validation binary_cross_entropy = 0.555385
Epoch 19
Loss = 6.5802e-01, PNorm = 63.5515, GNorm = 3.5610, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 0.431401
Epoch 20
Validation binary_cross_entropy = 0.434367
Epoch 21
Validation binary_cross_entropy = 0.566763
Epoch 22
Validation binary_cross_entropy = 0.576055
Epoch 23
Validation binary_cross_entropy = 0.375740
Epoch 24
Loss = 6.7032e-01, PNorm = 63.6515, GNorm = 3.1057, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 0.497830
Epoch 25
Validation binary_cross_entropy = 0.461870
Epoch 26
Validation binary_cross_entropy = 0.521824
Epoch 27
Validation binary_cross_entropy = 0.481439
Epoch 28
Validation binary_cross_entropy = 0.457925
Epoch 29
Loss = 6.1084e-01, PNorm = 63.7153, GNorm = 2.8922, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 0.472704
Epoch 30
Validation binary_cross_entropy = 0.517028
Epoch 31
Validation binary_cross_entropy = 0.486613
Epoch 32
Validation binary_cross_entropy = 0.485379
Epoch 33
Validation binary_cross_entropy = 0.468135
Epoch 34
Loss = 5.9811e-01, PNorm = 63.7809, GNorm = 2.4938, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 0.427763
Epoch 35
Validation binary_cross_entropy = 0.469912
Epoch 36
Validation binary_cross_entropy = 0.582947
Epoch 37
Validation binary_cross_entropy = 0.426217
Epoch 38
Validation binary_cross_entropy = 0.360635
Epoch 39
Loss = 6.4190e-01, PNorm = 63.8463, GNorm = 3.5083, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 0.423756
Epoch 40
Validation binary_cross_entropy = 0.641698
Epoch 41
Validation binary_cross_entropy = 0.456837
Epoch 42
Validation binary_cross_entropy = 0.351814
Epoch 43
Validation binary_cross_entropy = 0.423985
Epoch 44
Loss = 5.9507e-01, PNorm = 63.9060, GNorm = 2.3729, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 0.582152
Epoch 45
Validation binary_cross_entropy = 0.512781
Epoch 46
Validation binary_cross_entropy = 0.427940
Epoch 47
Validation binary_cross_entropy = 0.418260
Epoch 48
Validation binary_cross_entropy = 0.491310
Epoch 49
Loss = 6.2303e-01, PNorm = 63.9639, GNorm = 2.3490, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 0.479724
Epoch 50
Validation binary_cross_entropy = 0.441937
Epoch 51
Validation binary_cross_entropy = 0.442449
Epoch 52
Validation binary_cross_entropy = 0.424162
Epoch 53
Validation binary_cross_entropy = 0.467916
Epoch 54
Loss = 5.8621e-01, PNorm = 64.0248, GNorm = 1.7079, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 0.534935
Epoch 55
Validation binary_cross_entropy = 0.497515
Epoch 56
Validation binary_cross_entropy = 0.458755
Epoch 57
Validation binary_cross_entropy = 0.389588
Epoch 58
Validation binary_cross_entropy = 0.409997
Epoch 59
Loss = 5.8892e-01, PNorm = 64.0988, GNorm = 1.9613, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 0.407530
Epoch 60
Validation binary_cross_entropy = 0.524273
Epoch 61
Validation binary_cross_entropy = 0.432197
Epoch 62
Validation binary_cross_entropy = 0.456950
Epoch 63
Validation binary_cross_entropy = 0.508430
Epoch 64
Loss = 5.3164e-01, PNorm = 64.1781, GNorm = 1.6254, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 0.472443
Epoch 65
Validation binary_cross_entropy = 0.359730
Epoch 66
Validation binary_cross_entropy = 0.345241
Epoch 67
Validation binary_cross_entropy = 0.515456
Epoch 68
Validation binary_cross_entropy = 0.433147
Epoch 69
Loss = 4.9257e-01, PNorm = 64.2648, GNorm = 1.4643, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 0.365673
Epoch 70
Validation binary_cross_entropy = 0.412234
Epoch 71
Validation binary_cross_entropy = 0.438998
Epoch 72
Validation binary_cross_entropy = 0.335975
Epoch 73
Validation binary_cross_entropy = 0.418850
Epoch 74
Loss = 4.5810e-01, PNorm = 64.3710, GNorm = 1.7875, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 0.437598
Epoch 75
Validation binary_cross_entropy = 0.341689
Epoch 76
Validation binary_cross_entropy = 0.415183
Epoch 77
Validation binary_cross_entropy = 0.423582
Epoch 78
Validation binary_cross_entropy = 0.330176
Epoch 79
Loss = 4.6943e-01, PNorm = 64.4765, GNorm = 1.8497, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 0.450325
Epoch 80
Validation binary_cross_entropy = 0.415297
Epoch 81
Validation binary_cross_entropy = 0.322829
Epoch 82
Validation binary_cross_entropy = 0.406414
Epoch 83
Validation binary_cross_entropy = 0.519345
Epoch 84
Loss = 5.1618e-01, PNorm = 64.5854, GNorm = 1.7793, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 0.313922
Epoch 85
Validation binary_cross_entropy = 0.443533
Epoch 86
Validation binary_cross_entropy = 0.518932
Epoch 87
Validation binary_cross_entropy = 0.403173
Epoch 88
Validation binary_cross_entropy = 0.378945
Epoch 89
Loss = 4.0232e-01, PNorm = 64.7009, GNorm = 1.8410, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 0.460353
Epoch 90
Validation binary_cross_entropy = 0.357802
Epoch 91
Validation binary_cross_entropy = 0.298252
Epoch 92
Validation binary_cross_entropy = 0.688198
Epoch 93
Validation binary_cross_entropy = 0.446243
Epoch 94
Loss = 3.8489e-01, PNorm = 64.8121, GNorm = 1.4452, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 0.321528
Epoch 95
Validation binary_cross_entropy = 0.336616
Epoch 96
Validation binary_cross_entropy = 0.614482
Epoch 97
Validation binary_cross_entropy = 0.421732
Epoch 98
Validation binary_cross_entropy = 0.309133
Epoch 99
Loss = 3.8713e-01, PNorm = 64.9433, GNorm = 1.7246, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.412696
Epoch 100
Validation binary_cross_entropy = 0.487821
Epoch 101
Validation binary_cross_entropy = 0.368465
Epoch 102
Validation binary_cross_entropy = 0.354444
Epoch 103
Validation binary_cross_entropy = 0.438060
Epoch 104
Loss = 3.3578e-01, PNorm = 65.0739, GNorm = 1.5572, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.409859
Epoch 105
Validation binary_cross_entropy = 0.347598
Epoch 106
Validation binary_cross_entropy = 0.414044
Epoch 107
Validation binary_cross_entropy = 0.397244
Epoch 108
Validation binary_cross_entropy = 0.334173
Epoch 109
Loss = 3.3298e-01, PNorm = 65.1985, GNorm = 2.6815, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.513396
Epoch 110
Validation binary_cross_entropy = 0.483032
Epoch 111
Validation binary_cross_entropy = 0.370771
Epoch 112
Validation binary_cross_entropy = 0.475437
Epoch 113
Validation binary_cross_entropy = 0.522656
Epoch 114
Loss = 3.4429e-01, PNorm = 65.3315, GNorm = 1.3007, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.379143
Epoch 115
Validation binary_cross_entropy = 0.430793
Epoch 116
Validation binary_cross_entropy = 0.527545
Epoch 117
Validation binary_cross_entropy = 0.359518
Epoch 118
Validation binary_cross_entropy = 0.321336
Epoch 119
Loss = 4.1713e-01, PNorm = 65.4728, GNorm = 1.4663, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.378599
Epoch 120
Validation binary_cross_entropy = 0.492115
Epoch 121
Validation binary_cross_entropy = 0.388650
Epoch 122
Validation binary_cross_entropy = 0.424830
Epoch 123
Validation binary_cross_entropy = 0.495568
Epoch 124
Loss = 2.6186e-01, PNorm = 65.6155, GNorm = 1.0727, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.512523
Epoch 125
Validation binary_cross_entropy = 0.550059
Epoch 126
Validation binary_cross_entropy = 0.543183
Epoch 127
Validation binary_cross_entropy = 0.542042
Epoch 128
Validation binary_cross_entropy = 0.507629
Epoch 129
Loss = 1.7519e-01, PNorm = 65.7447, GNorm = 1.4237, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.488613
Epoch 130
Validation binary_cross_entropy = 0.484101
Epoch 131
Validation binary_cross_entropy = 0.480331
Epoch 132
Validation binary_cross_entropy = 0.526641
Epoch 133
Validation binary_cross_entropy = 0.642370
Epoch 134
Loss = 2.8842e-01, PNorm = 65.8746, GNorm = 3.5129, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.527713
Epoch 135
Validation binary_cross_entropy = 0.502124
Epoch 136
Validation binary_cross_entropy = 0.554298
Epoch 137
Validation binary_cross_entropy = 0.489662
Epoch 138
Validation binary_cross_entropy = 0.496444
Epoch 139
Loss = 2.2473e-01, PNorm = 66.0105, GNorm = 1.1665, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.578672
Epoch 140
Validation binary_cross_entropy = 0.674068
Epoch 141
Validation binary_cross_entropy = 0.579720
Epoch 142
Validation binary_cross_entropy = 0.600954
Epoch 143
Validation binary_cross_entropy = 0.671611
Epoch 144
Loss = 1.8399e-01, PNorm = 66.1370, GNorm = 1.9431, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.705063
Epoch 145
Validation binary_cross_entropy = 0.745262
Epoch 146
Validation binary_cross_entropy = 0.791273
Epoch 147
Validation binary_cross_entropy = 0.786166
Epoch 148
Validation binary_cross_entropy = 0.715535
Epoch 149
Loss = 1.6662e-01, PNorm = 66.2380, GNorm = 3.7739, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.675608
Epoch 150
Validation binary_cross_entropy = 0.587579
Epoch 151
Validation binary_cross_entropy = 0.535914
Epoch 152
Validation binary_cross_entropy = 0.518489
Epoch 153
Validation binary_cross_entropy = 0.515610
Epoch 154
Loss = 1.7910e-01, PNorm = 66.3425, GNorm = 1.7638, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.554122
Epoch 155
Validation binary_cross_entropy = 0.570995
Epoch 156
Validation binary_cross_entropy = 0.605536
Epoch 157
Validation binary_cross_entropy = 0.608824
Epoch 158
Validation binary_cross_entropy = 0.664970
Epoch 159
Loss = 1.8547e-01, PNorm = 66.4384, GNorm = 3.0301, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.659517
Epoch 160
Validation binary_cross_entropy = 0.678668
Epoch 161
Validation binary_cross_entropy = 0.698399
Epoch 162
Validation binary_cross_entropy = 0.696986
Epoch 163
Validation binary_cross_entropy = 0.682270
Epoch 164
Loss = 1.3554e-01, PNorm = 66.5338, GNorm = 3.5417, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.699160
Epoch 165
Validation binary_cross_entropy = 0.759140
Epoch 166
Validation binary_cross_entropy = 0.697140
Epoch 167
Validation binary_cross_entropy = 0.667196
Epoch 168
Validation binary_cross_entropy = 0.706802
Epoch 169
Loss = 1.8584e-01, PNorm = 66.6263, GNorm = 0.6898, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.774440
Epoch 170
Validation binary_cross_entropy = 0.774215
Epoch 171
Validation binary_cross_entropy = 0.745034
Epoch 172
Validation binary_cross_entropy = 0.737186
Epoch 173
Validation binary_cross_entropy = 0.786655
Epoch 174
Loss = 1.4078e-01, PNorm = 66.7301, GNorm = 1.3500, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 0.806647
Epoch 175
Validation binary_cross_entropy = 0.800242
Epoch 176
Validation binary_cross_entropy = 0.921135
Epoch 177
Validation binary_cross_entropy = 0.701197
Epoch 178
Validation binary_cross_entropy = 0.594028
Epoch 179
Loss = 1.3529e-01, PNorm = 66.8368, GNorm = 1.0233, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 0.568053
Epoch 180
Validation binary_cross_entropy = 0.536981
Epoch 181
Validation binary_cross_entropy = 0.557621
Epoch 182
Validation binary_cross_entropy = 0.596732
Epoch 183
Validation binary_cross_entropy = 0.630400
Epoch 184
Loss = 9.7319e-02, PNorm = 66.9607, GNorm = 1.3460, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 0.683154
Epoch 185
Validation binary_cross_entropy = 0.720070
Epoch 186
Validation binary_cross_entropy = 0.752898
Epoch 187
Validation binary_cross_entropy = 0.803656
Epoch 188
Validation binary_cross_entropy = 0.788732
Epoch 189
Loss = 6.4205e-02, PNorm = 67.0530, GNorm = 1.7694, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 0.864521
Epoch 190
Validation binary_cross_entropy = 0.801609
Epoch 191
Validation binary_cross_entropy = 0.835918
Epoch 192
Validation binary_cross_entropy = 0.799484
Epoch 193
Validation binary_cross_entropy = 0.710151
Epoch 194
Loss = 1.2747e-01, PNorm = 67.1372, GNorm = 2.8043, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 0.713651
Epoch 195
Validation binary_cross_entropy = 0.671399
Epoch 196
Validation binary_cross_entropy = 0.731035
Epoch 197
Validation binary_cross_entropy = 0.731456
Epoch 198
Validation binary_cross_entropy = 0.703867
Epoch 199
Loss = 2.2353e-01, PNorm = 67.2191, GNorm = 4.1797, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 0.735032
Epoch 200
Validation binary_cross_entropy = 0.875234
Epoch 201
Validation binary_cross_entropy = 0.845570
Epoch 202
Validation binary_cross_entropy = 0.729508
Epoch 203
Validation binary_cross_entropy = 0.694990
Epoch 204
Loss = 1.1624e-01, PNorm = 67.3057, GNorm = 1.7970, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 0.709775
Epoch 205
Validation binary_cross_entropy = 0.673603
Epoch 206
Validation binary_cross_entropy = 0.686100
Epoch 207
Validation binary_cross_entropy = 0.640591
Epoch 208
Validation binary_cross_entropy = 0.622091
Epoch 209
Loss = 8.8846e-02, PNorm = 67.3843, GNorm = 1.1637, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 0.629825
Epoch 210
Validation binary_cross_entropy = 0.669578
Epoch 211
Validation binary_cross_entropy = 0.714685
Epoch 212
Validation binary_cross_entropy = 0.712062
Epoch 213
Validation binary_cross_entropy = 0.733969
Epoch 214
Loss = 1.2771e-01, PNorm = 67.4633, GNorm = 0.7641, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 0.775567
Epoch 215
Validation binary_cross_entropy = 0.808647
Epoch 216
Validation binary_cross_entropy = 0.854152
Epoch 217
Validation binary_cross_entropy = 0.845440
Epoch 218
Validation binary_cross_entropy = 0.766352
Epoch 219
Loss = 6.2457e-02, PNorm = 67.5301, GNorm = 1.2830, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 0.751419
Epoch 220
Validation binary_cross_entropy = 0.756421
Epoch 221
Validation binary_cross_entropy = 0.770285
Epoch 222
Validation binary_cross_entropy = 0.796439
Epoch 223
Validation binary_cross_entropy = 0.810559
Epoch 224
Loss = 6.6946e-02, PNorm = 67.5919, GNorm = 0.7677, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 0.821371
Epoch 225
Validation binary_cross_entropy = 0.856219
Epoch 226
Validation binary_cross_entropy = 0.921635
Epoch 227
Validation binary_cross_entropy = 0.988436
Epoch 228
Validation binary_cross_entropy = 1.015962
Epoch 229
Loss = 5.4501e-02, PNorm = 67.6420, GNorm = 1.6215, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 1.064670
Epoch 230
Validation binary_cross_entropy = 1.100842
Epoch 231
Validation binary_cross_entropy = 1.110163
Epoch 232
Validation binary_cross_entropy = 1.103050
Epoch 233
Validation binary_cross_entropy = 1.117529
Epoch 234
Loss = 7.8469e-02, PNorm = 67.6957, GNorm = 2.0957, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 0.979556
Epoch 235
Validation binary_cross_entropy = 0.891325
Epoch 236
Validation binary_cross_entropy = 0.824478
Epoch 237
Validation binary_cross_entropy = 0.802508
Epoch 238
Validation binary_cross_entropy = 0.815035
Epoch 239
Loss = 1.4276e-01, PNorm = 67.7460, GNorm = 1.5064, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 0.753825
Epoch 240
Validation binary_cross_entropy = 0.703224
Epoch 241
Validation binary_cross_entropy = 0.715579
Epoch 242
Validation binary_cross_entropy = 0.769384
Epoch 243
Validation binary_cross_entropy = 0.816311
Epoch 244
Loss = 9.6183e-02, PNorm = 67.8153, GNorm = 0.8879, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 0.879800
Epoch 245
Validation binary_cross_entropy = 0.951524
Epoch 246
Validation binary_cross_entropy = 0.981683
Epoch 247
Validation binary_cross_entropy = 1.051914
Epoch 248
Validation binary_cross_entropy = 1.095258
Epoch 249
Loss = 4.7169e-02, PNorm = 67.8732, GNorm = 0.4464, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 1.024466
Epoch 250
Validation binary_cross_entropy = 1.004154
Epoch 251
Validation binary_cross_entropy = 0.967364
Epoch 252
Validation binary_cross_entropy = 0.908230
Epoch 253
Validation binary_cross_entropy = 0.852860
Epoch 254
Loss = 6.3485e-02, PNorm = 67.9236, GNorm = 1.0660, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 0.820879
Epoch 255
Validation binary_cross_entropy = 0.795674
Epoch 256
Validation binary_cross_entropy = 0.789801
Epoch 257
Validation binary_cross_entropy = 0.793679
Epoch 258
Validation binary_cross_entropy = 0.820466
Epoch 259
Loss = 9.9206e-02, PNorm = 67.9862, GNorm = 2.0106, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 0.887989
Epoch 260
Validation binary_cross_entropy = 0.916336
Epoch 261
Validation binary_cross_entropy = 0.916691
Epoch 262
Validation binary_cross_entropy = 0.925399
Epoch 263
Validation binary_cross_entropy = 0.965837
Epoch 264
Loss = 1.1548e-01, PNorm = 68.0462, GNorm = 3.4384, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 0.973522
Epoch 265
Validation binary_cross_entropy = 1.023698
Epoch 266
Validation binary_cross_entropy = 1.039478
Epoch 267
Validation binary_cross_entropy = 1.005325
Epoch 268
Validation binary_cross_entropy = 1.015441
Epoch 269
Loss = 6.5457e-02, PNorm = 68.0935, GNorm = 1.0933, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 1.009015
Epoch 270
Validation binary_cross_entropy = 1.012466
Epoch 271
Validation binary_cross_entropy = 1.006424
Epoch 272
Validation binary_cross_entropy = 0.961708
Epoch 273
Validation binary_cross_entropy = 0.943437
Epoch 274
Loss = 6.7687e-02, PNorm = 68.1480, GNorm = 0.8594, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 0.930276
Epoch 275
Validation binary_cross_entropy = 0.929971
Epoch 276
Validation binary_cross_entropy = 0.924498
Epoch 277
Validation binary_cross_entropy = 0.926440
Epoch 278
Validation binary_cross_entropy = 0.926770
Epoch 279
Loss = 6.4921e-02, PNorm = 68.2038, GNorm = 1.6578, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 1.014007
Epoch 280
Validation binary_cross_entropy = 0.942076
Epoch 281
Validation binary_cross_entropy = 1.022825
Epoch 282
Validation binary_cross_entropy = 0.938791
Epoch 283
Validation binary_cross_entropy = 0.919641
Epoch 284
Loss = 8.2633e-02, PNorm = 68.2679, GNorm = 2.9245, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 0.880076
Epoch 285
Validation binary_cross_entropy = 0.867580
Epoch 286
Validation binary_cross_entropy = 0.888576
Epoch 287
Validation binary_cross_entropy = 0.794869
Epoch 288
Validation binary_cross_entropy = 0.758853
Epoch 289
Loss = 8.0506e-02, PNorm = 68.3306, GNorm = 0.9081, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 0.747756
Epoch 290
Validation binary_cross_entropy = 0.727784
Epoch 291
Validation binary_cross_entropy = 0.755987
Epoch 292
Validation binary_cross_entropy = 0.776505
Epoch 293
Validation binary_cross_entropy = 0.787216
Epoch 294
Loss = 2.4763e-02, PNorm = 68.3990, GNorm = 0.8119, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 0.834675
Epoch 295
Validation binary_cross_entropy = 0.865502
Epoch 296
Validation binary_cross_entropy = 0.914333
Epoch 297
Validation binary_cross_entropy = 0.958716
Epoch 298
Validation binary_cross_entropy = 0.988244
Epoch 299
Loss = 5.5585e-02, PNorm = 68.4531, GNorm = 0.7104, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 0.999965
Model 0 best validation binary_cross_entropy = 0.287578 on epoch 0
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.223729
Ensemble test binary_cross_entropy = 0.223729
Fold 8
Splitting data with seed 8
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.4, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=800, bias=False)
        (W_h): Linear(in_features=800, out_features=800, bias=False)
        (W_o): Linear(in_features=933, out_features=800, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.4, inplace=False)
    (1): Linear(in_features=3100, out_features=800, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.4, inplace=False)
    (4): Linear(in_features=800, out_features=800, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.4, inplace=False)
    (7): Linear(in_features=800, out_features=1, bias=True)
  )
)
Number of parameters = 4,627,201
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 0.287546
Epoch 1
Validation binary_cross_entropy = 0.301045
Epoch 2
Validation binary_cross_entropy = 0.583419
Epoch 3
Validation binary_cross_entropy = 0.319260
Epoch 4
Loss = 8.4577e-01, PNorm = 63.1524, GNorm = 7.6351, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 0.297064
Epoch 5
Validation binary_cross_entropy = 0.525440
Epoch 6
Validation binary_cross_entropy = 0.813832
Epoch 7
Validation binary_cross_entropy = 0.322716
Epoch 8
Validation binary_cross_entropy = 0.430601
Epoch 9
Loss = 6.5036e-01, PNorm = 63.2942, GNorm = 4.3828, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 0.589930
Epoch 10
Validation binary_cross_entropy = 0.347977
Epoch 11
Validation binary_cross_entropy = 0.523647
Epoch 12
Validation binary_cross_entropy = 0.731850
Epoch 13
Validation binary_cross_entropy = 0.302254
Epoch 14
Loss = 9.3429e-01, PNorm = 63.4532, GNorm = 6.9298, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 0.343261
Epoch 15
Validation binary_cross_entropy = 0.570982
Epoch 16
Validation binary_cross_entropy = 0.376336
Epoch 17
Validation binary_cross_entropy = 0.437099
Epoch 18
Validation binary_cross_entropy = 0.617986
Epoch 19
Loss = 7.1439e-01, PNorm = 63.5767, GNorm = 3.5800, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 0.366876
Epoch 20
Validation binary_cross_entropy = 0.333799
Epoch 21
Validation binary_cross_entropy = 0.654548
Epoch 22
Validation binary_cross_entropy = 0.631589
Epoch 23
Validation binary_cross_entropy = 0.338761
Epoch 24
Loss = 7.0166e-01, PNorm = 63.6766, GNorm = 4.7686, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 0.346973
Epoch 25
Validation binary_cross_entropy = 0.618859
Epoch 26
Validation binary_cross_entropy = 0.695850
Epoch 27
Validation binary_cross_entropy = 0.450053
Epoch 28
Validation binary_cross_entropy = 0.402421
Epoch 29
Loss = 5.9487e-01, PNorm = 63.7513, GNorm = 2.6708, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 0.571627
Epoch 30
Validation binary_cross_entropy = 0.708425
Epoch 31
Validation binary_cross_entropy = 0.513279
Epoch 32
Validation binary_cross_entropy = 0.401880
Epoch 33
Validation binary_cross_entropy = 0.388981
Epoch 34
Loss = 6.0567e-01, PNorm = 63.8215, GNorm = 3.1519, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 0.544627
Epoch 35
Validation binary_cross_entropy = 0.651220
Epoch 36
Validation binary_cross_entropy = 0.560333
Epoch 37
Validation binary_cross_entropy = 0.471793
Epoch 38
Validation binary_cross_entropy = 0.491366
Epoch 39
Loss = 6.4089e-01, PNorm = 63.8817, GNorm = 2.5876, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 0.584887
Epoch 40
Validation binary_cross_entropy = 0.602236
Epoch 41
Validation binary_cross_entropy = 0.543659
Epoch 42
Validation binary_cross_entropy = 0.472970
Epoch 43
Validation binary_cross_entropy = 0.478900
Epoch 44
Loss = 6.5011e-01, PNorm = 63.9424, GNorm = 2.2508, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 0.502020
Epoch 45
Validation binary_cross_entropy = 0.531290
Epoch 46
Validation binary_cross_entropy = 0.605189
Epoch 47
Validation binary_cross_entropy = 0.578180
Epoch 48
Validation binary_cross_entropy = 0.472469
Epoch 49
Loss = 5.6602e-01, PNorm = 64.0065, GNorm = 2.1797, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 0.445336
Epoch 50
Validation binary_cross_entropy = 0.601404
Epoch 51
Validation binary_cross_entropy = 0.527437
Epoch 52
Validation binary_cross_entropy = 0.444580
Epoch 53
Validation binary_cross_entropy = 0.518978
Epoch 54
Loss = 5.0561e-01, PNorm = 64.0863, GNorm = 2.3914, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 0.513229
Epoch 55
Validation binary_cross_entropy = 0.416871
Epoch 56
Validation binary_cross_entropy = 0.385323
Epoch 57
Validation binary_cross_entropy = 0.439685
Epoch 58
Validation binary_cross_entropy = 0.477131
Epoch 59
Loss = 5.2129e-01, PNorm = 64.1718, GNorm = 1.8437, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 0.375356
Epoch 60
Validation binary_cross_entropy = 0.405288
Epoch 61
Validation binary_cross_entropy = 0.500979
Epoch 62
Validation binary_cross_entropy = 0.373644
Epoch 63
Validation binary_cross_entropy = 0.492720
Epoch 64
Loss = 4.4872e-01, PNorm = 64.2654, GNorm = 1.8776, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 0.475682
Epoch 65
Validation binary_cross_entropy = 0.407719
Epoch 66
Validation binary_cross_entropy = 0.432721
Epoch 67
Validation binary_cross_entropy = 0.347950
Epoch 68
Validation binary_cross_entropy = 0.510376
Epoch 69
Loss = 5.1953e-01, PNorm = 64.3588, GNorm = 1.8400, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 0.306464
Epoch 70
Validation binary_cross_entropy = 0.471980
Epoch 71
Validation binary_cross_entropy = 0.507622
Epoch 72
Validation binary_cross_entropy = 0.364043
Epoch 73
Validation binary_cross_entropy = 0.382895
Epoch 74
Loss = 4.3330e-01, PNorm = 64.4643, GNorm = 2.4359, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 0.396784
Epoch 75
Validation binary_cross_entropy = 0.327292
Epoch 76
Validation binary_cross_entropy = 0.387187
Epoch 77
Validation binary_cross_entropy = 0.335374
Epoch 78
Validation binary_cross_entropy = 0.425642
Epoch 79
Loss = 4.3273e-01, PNorm = 64.5663, GNorm = 1.8838, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 0.455353
Epoch 80
Validation binary_cross_entropy = 0.370196
Epoch 81
Validation binary_cross_entropy = 0.391624
Epoch 82
Validation binary_cross_entropy = 0.372607
Epoch 83
Validation binary_cross_entropy = 0.331110
Epoch 84
Loss = 3.4628e-01, PNorm = 64.6883, GNorm = 1.7338, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 0.508530
Epoch 85
Validation binary_cross_entropy = 0.338244
Epoch 86
Validation binary_cross_entropy = 0.408943
Epoch 87
Validation binary_cross_entropy = 0.457768
Epoch 88
Validation binary_cross_entropy = 0.318494
Epoch 89
Loss = 4.5395e-01, PNorm = 64.8013, GNorm = 2.4420, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 0.482693
Epoch 90
Validation binary_cross_entropy = 0.375504
Epoch 91
Validation binary_cross_entropy = 0.310761
Epoch 92
Validation binary_cross_entropy = 0.521798
Epoch 93
Validation binary_cross_entropy = 0.321695
Epoch 94
Loss = 4.8637e-01, PNorm = 64.9364, GNorm = 4.4356, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 0.308631
Epoch 95
Validation binary_cross_entropy = 0.458235
Epoch 96
Validation binary_cross_entropy = 0.425203
Epoch 97
Validation binary_cross_entropy = 0.341972
Epoch 98
Validation binary_cross_entropy = 0.410270
Epoch 99
Loss = 3.4311e-01, PNorm = 65.0810, GNorm = 1.9150, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.517458
Epoch 100
Validation binary_cross_entropy = 0.347972
Epoch 101
Validation binary_cross_entropy = 0.395681
Epoch 102
Validation binary_cross_entropy = 0.401147
Epoch 103
Validation binary_cross_entropy = 0.346039
Epoch 104
Loss = 2.8668e-01, PNorm = 65.2046, GNorm = 1.5033, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.396467
Epoch 105
Validation binary_cross_entropy = 0.425795
Epoch 106
Validation binary_cross_entropy = 0.441654
Epoch 107
Validation binary_cross_entropy = 0.519421
Epoch 108
Validation binary_cross_entropy = 0.465037
Epoch 109
Loss = 2.7748e-01, PNorm = 65.3302, GNorm = 2.1324, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.522512
Epoch 110
Validation binary_cross_entropy = 0.335623
Epoch 111
Validation binary_cross_entropy = 0.290490
Epoch 112
Validation binary_cross_entropy = 0.365171
Epoch 113
Validation binary_cross_entropy = 0.524182
Epoch 114
Loss = 5.0223e-01, PNorm = 65.4578, GNorm = 1.6733, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.336200
Epoch 115
Validation binary_cross_entropy = 0.297966
Epoch 116
Validation binary_cross_entropy = 0.339121
Epoch 117
Validation binary_cross_entropy = 0.435102
Epoch 118
Validation binary_cross_entropy = 0.436798
Epoch 119
Loss = 2.9040e-01, PNorm = 65.5984, GNorm = 2.1362, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.489594
Epoch 120
Validation binary_cross_entropy = 0.499471
Epoch 121
Validation binary_cross_entropy = 0.494762
Epoch 122
Validation binary_cross_entropy = 0.456593
Epoch 123
Validation binary_cross_entropy = 0.440197
Epoch 124
Loss = 2.0595e-01, PNorm = 65.7118, GNorm = 1.5369, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.405393
Epoch 125
Validation binary_cross_entropy = 0.411530
Epoch 126
Validation binary_cross_entropy = 0.423887
Epoch 127
Validation binary_cross_entropy = 0.446198
Epoch 128
Validation binary_cross_entropy = 0.481814
Epoch 129
Loss = 1.4175e-01, PNorm = 65.8289, GNorm = 1.7824, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.521217
Epoch 130
Validation binary_cross_entropy = 0.537195
Epoch 131
Validation binary_cross_entropy = 0.512177
Epoch 132
Validation binary_cross_entropy = 0.504164
Epoch 133
Validation binary_cross_entropy = 0.484348
Epoch 134
Loss = 1.5122e-01, PNorm = 65.9322, GNorm = 2.6983, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.498956
Epoch 135
Validation binary_cross_entropy = 0.435979
Epoch 136
Validation binary_cross_entropy = 0.430129
Epoch 137
Validation binary_cross_entropy = 0.484654
Epoch 138
Validation binary_cross_entropy = 0.455105
Epoch 139
Loss = 1.8520e-01, PNorm = 66.0361, GNorm = 5.7245, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.463649
Epoch 140
Validation binary_cross_entropy = 0.518297
Epoch 141
Validation binary_cross_entropy = 0.485850
Epoch 142
Validation binary_cross_entropy = 0.527778
Epoch 143
Validation binary_cross_entropy = 0.513994
Epoch 144
Loss = 1.9564e-01, PNorm = 66.1431, GNorm = 2.5383, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.617826
Epoch 145
Validation binary_cross_entropy = 0.521381
Epoch 146
Validation binary_cross_entropy = 0.532412
Epoch 147
Validation binary_cross_entropy = 0.519980
Epoch 148
Validation binary_cross_entropy = 0.495770
Epoch 149
Loss = 1.7205e-01, PNorm = 66.2447, GNorm = 0.9299, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.506710
Epoch 150
Validation binary_cross_entropy = 0.531751
Epoch 151
Validation binary_cross_entropy = 0.569683
Epoch 152
Validation binary_cross_entropy = 0.585215
Epoch 153
Validation binary_cross_entropy = 0.612885
Epoch 154
Loss = 1.2968e-01, PNorm = 66.3352, GNorm = 2.6267, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.620807
Epoch 155
Validation binary_cross_entropy = 0.623851
Epoch 156
Validation binary_cross_entropy = 0.637385
Epoch 157
Validation binary_cross_entropy = 0.650803
Epoch 158
Validation binary_cross_entropy = 0.676008
Epoch 159
Loss = 1.1395e-01, PNorm = 66.4153, GNorm = 1.3483, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.712153
Epoch 160
Validation binary_cross_entropy = 0.715740
Epoch 161
Validation binary_cross_entropy = 0.700757
Epoch 162
Validation binary_cross_entropy = 0.693427
Epoch 163
Validation binary_cross_entropy = 0.689762
Epoch 164
Loss = 1.3241e-01, PNorm = 66.4816, GNorm = 1.3883, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.665492
Epoch 165
Validation binary_cross_entropy = 0.640673
Epoch 166
Validation binary_cross_entropy = 0.649005
Epoch 167
Validation binary_cross_entropy = 0.670653
Epoch 168
Validation binary_cross_entropy = 0.688505
Epoch 169
Loss = 1.4489e-01, PNorm = 66.5509, GNorm = 1.3800, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.697485
Epoch 170
Validation binary_cross_entropy = 0.699114
Epoch 171
Validation binary_cross_entropy = 0.713904
Epoch 172
Validation binary_cross_entropy = 0.726096
Epoch 173
Validation binary_cross_entropy = 0.746595
Epoch 174
Loss = 7.4059e-02, PNorm = 66.6248, GNorm = 0.9997, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 0.812994
Epoch 175
Validation binary_cross_entropy = 0.866543
Epoch 176
Validation binary_cross_entropy = 0.880532
Epoch 177
Validation binary_cross_entropy = 0.920976
Epoch 178
Validation binary_cross_entropy = 0.941716
Epoch 179
Loss = 9.5584e-02, PNorm = 66.6998, GNorm = 1.1224, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 0.953082
Epoch 180
Validation binary_cross_entropy = 1.002095
Epoch 181
Validation binary_cross_entropy = 0.979125
Epoch 182
Validation binary_cross_entropy = 0.886137
Epoch 183
Validation binary_cross_entropy = 0.795813
Epoch 184
Loss = 9.4787e-02, PNorm = 66.7664, GNorm = 3.9095, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 0.780428
Epoch 185
Validation binary_cross_entropy = 0.774053
Epoch 186
Validation binary_cross_entropy = 0.747493
Epoch 187
Validation binary_cross_entropy = 0.785110
Epoch 188
Validation binary_cross_entropy = 0.797144
Epoch 189
Loss = 1.6604e-01, PNorm = 66.8575, GNorm = 1.0933, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 0.812701
Epoch 190
Validation binary_cross_entropy = 0.793171
Epoch 191
Validation binary_cross_entropy = 0.781198
Epoch 192
Validation binary_cross_entropy = 0.787704
Epoch 193
Validation binary_cross_entropy = 0.864741
Epoch 194
Loss = 1.0508e-01, PNorm = 66.9352, GNorm = 1.1071, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 0.841284
Epoch 195
Validation binary_cross_entropy = 0.828268
Epoch 196
Validation binary_cross_entropy = 0.821455
Epoch 197
Validation binary_cross_entropy = 0.849786
Epoch 198
Validation binary_cross_entropy = 0.828047
Epoch 199
Loss = 1.5126e-01, PNorm = 66.9967, GNorm = 1.6932, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 0.860740
Epoch 200
Validation binary_cross_entropy = 0.840062
Epoch 201
Validation binary_cross_entropy = 0.823786
Epoch 202
Validation binary_cross_entropy = 0.836618
Epoch 203
Validation binary_cross_entropy = 0.847698
Epoch 204
Loss = 9.2348e-02, PNorm = 67.0542, GNorm = 3.7293, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 0.810633
Epoch 205
Validation binary_cross_entropy = 0.788968
Epoch 206
Validation binary_cross_entropy = 0.811492
Epoch 207
Validation binary_cross_entropy = 0.827364
Epoch 208
Validation binary_cross_entropy = 0.896396
Epoch 209
Loss = 5.1363e-02, PNorm = 67.1177, GNorm = 1.5731, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 0.979551
Epoch 210
Validation binary_cross_entropy = 0.998246
Epoch 211
Validation binary_cross_entropy = 1.003611
Epoch 212
Validation binary_cross_entropy = 1.023584
Epoch 213
Validation binary_cross_entropy = 0.996808
Epoch 214
Loss = 3.8779e-02, PNorm = 67.1723, GNorm = 1.7164, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 0.994138
Epoch 215
Validation binary_cross_entropy = 0.992382
Epoch 216
Validation binary_cross_entropy = 1.032755
Epoch 217
Validation binary_cross_entropy = 1.046893
Epoch 218
Validation binary_cross_entropy = 1.087404
Epoch 219
Loss = 4.1366e-02, PNorm = 67.2220, GNorm = 0.8613, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 1.198201
Epoch 220
Validation binary_cross_entropy = 1.146503
Epoch 221
Validation binary_cross_entropy = 1.096649
Epoch 222
Validation binary_cross_entropy = 1.004526
Epoch 223
Validation binary_cross_entropy = 1.060539
Epoch 224
Loss = 1.5561e-01, PNorm = 67.2725, GNorm = 5.9531, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 0.984987
Epoch 225
Validation binary_cross_entropy = 0.860323
Epoch 226
Validation binary_cross_entropy = 0.796541
Epoch 227
Validation binary_cross_entropy = 0.799652
Epoch 228
Validation binary_cross_entropy = 0.830485
Epoch 229
Loss = 6.5051e-02, PNorm = 67.3385, GNorm = 0.8570, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 0.756934
Epoch 230
Validation binary_cross_entropy = 0.734597
Epoch 231
Validation binary_cross_entropy = 0.750151
Epoch 232
Validation binary_cross_entropy = 0.853515
Epoch 233
Validation binary_cross_entropy = 0.899295
Epoch 234
Loss = 5.9383e-02, PNorm = 67.4004, GNorm = 2.1420, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 0.889633
Epoch 235
Validation binary_cross_entropy = 0.933429
Epoch 236
Validation binary_cross_entropy = 0.978365
Epoch 237
Validation binary_cross_entropy = 1.199200
Epoch 238
Validation binary_cross_entropy = 0.878247
Epoch 239
Loss = 1.9607e-01, PNorm = 67.4600, GNorm = 2.8993, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 0.767469
Epoch 240
Validation binary_cross_entropy = 0.590216
Epoch 241
Validation binary_cross_entropy = 0.523086
Epoch 242
Validation binary_cross_entropy = 0.492568
Epoch 243
Validation binary_cross_entropy = 0.475630
Epoch 244
Loss = 1.9466e-01, PNorm = 67.5315, GNorm = 1.4084, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 0.509687
Epoch 245
Validation binary_cross_entropy = 0.555092
Epoch 246
Validation binary_cross_entropy = 0.585116
Epoch 247
Validation binary_cross_entropy = 0.716313
Epoch 248
Validation binary_cross_entropy = 0.831888
Epoch 249
Loss = 1.8624e-01, PNorm = 67.6102, GNorm = 4.6866, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 0.799740
Epoch 250
Validation binary_cross_entropy = 0.766069
Epoch 251
Validation binary_cross_entropy = 0.798226
Epoch 252
Validation binary_cross_entropy = 0.828309
Epoch 253
Validation binary_cross_entropy = 0.880801
Epoch 254
Loss = 7.7348e-02, PNorm = 67.6752, GNorm = 2.4505, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 0.941758
Epoch 255
Validation binary_cross_entropy = 0.903260
Epoch 256
Validation binary_cross_entropy = 0.894290
Epoch 257
Validation binary_cross_entropy = 0.906302
Epoch 258
Validation binary_cross_entropy = 0.935907
Epoch 259
Loss = 6.8140e-02, PNorm = 67.7337, GNorm = 1.5150, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 1.007627
Epoch 260
Validation binary_cross_entropy = 0.970751
Epoch 261
Validation binary_cross_entropy = 0.899228
Epoch 262
Validation binary_cross_entropy = 0.891010
Epoch 263
Validation binary_cross_entropy = 0.901733
Epoch 264
Loss = 8.8919e-02, PNorm = 67.7765, GNorm = 1.5320, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 0.913651
Epoch 265
Validation binary_cross_entropy = 0.864576
Epoch 266
Validation binary_cross_entropy = 0.867519
Epoch 267
Validation binary_cross_entropy = 0.886764
Epoch 268
Validation binary_cross_entropy = 0.912198
Epoch 269
Loss = 4.6500e-02, PNorm = 67.8294, GNorm = 0.6394, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 0.983108
Epoch 270
Validation binary_cross_entropy = 1.062547
Epoch 271
Validation binary_cross_entropy = 1.063928
Epoch 272
Validation binary_cross_entropy = 1.075137
Epoch 273
Validation binary_cross_entropy = 1.145001
Epoch 274
Loss = 8.5132e-02, PNorm = 67.8737, GNorm = 1.5733, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 1.094972
Epoch 275
Validation binary_cross_entropy = 1.053515
Epoch 276
Validation binary_cross_entropy = 1.049752
Epoch 277
Validation binary_cross_entropy = 1.052662
Epoch 278
Validation binary_cross_entropy = 1.061819
Epoch 279
Loss = 1.4937e-02, PNorm = 67.9177, GNorm = 0.3675, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 1.079000
Epoch 280
Validation binary_cross_entropy = 1.128458
Epoch 281
Validation binary_cross_entropy = 1.166914
Epoch 282
Validation binary_cross_entropy = 1.153983
Epoch 283
Validation binary_cross_entropy = 1.138781
Epoch 284
Loss = 5.3274e-02, PNorm = 67.9481, GNorm = 1.3720, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 1.133167
Epoch 285
Validation binary_cross_entropy = 1.154596
Epoch 286
Validation binary_cross_entropy = 1.093543
Epoch 287
Validation binary_cross_entropy = 1.023464
Epoch 288
Validation binary_cross_entropy = 1.006403
Epoch 289
Loss = 2.4342e-02, PNorm = 67.9829, GNorm = 1.6654, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 1.008850
Epoch 290
Validation binary_cross_entropy = 1.018823
Epoch 291
Validation binary_cross_entropy = 1.037742
Epoch 292
Validation binary_cross_entropy = 1.036773
Epoch 293
Validation binary_cross_entropy = 1.043579
Epoch 294
Loss = 7.0212e-02, PNorm = 68.0159, GNorm = 1.0527, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 1.067984
Epoch 295
Validation binary_cross_entropy = 1.085332
Epoch 296
Validation binary_cross_entropy = 1.102294
Epoch 297
Validation binary_cross_entropy = 1.114647
Epoch 298
Validation binary_cross_entropy = 1.204197
Epoch 299
Loss = 5.7316e-02, PNorm = 68.0507, GNorm = 3.8868, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 1.227961
Model 0 best validation binary_cross_entropy = 0.287546 on epoch 0
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.225327
Ensemble test binary_cross_entropy = 0.225327
Fold 9
Splitting data with seed 9
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.4, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=800, bias=False)
        (W_h): Linear(in_features=800, out_features=800, bias=False)
        (W_o): Linear(in_features=933, out_features=800, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.4, inplace=False)
    (1): Linear(in_features=3100, out_features=800, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.4, inplace=False)
    (4): Linear(in_features=800, out_features=800, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.4, inplace=False)
    (7): Linear(in_features=800, out_features=1, bias=True)
  )
)
Number of parameters = 4,627,201
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 0.288900
Epoch 1
Validation binary_cross_entropy = 0.296113
Epoch 2
Validation binary_cross_entropy = 0.627527
Epoch 3
Validation binary_cross_entropy = 0.325645
Epoch 4
Loss = 8.3044e-01, PNorm = 63.1499, GNorm = 6.1946, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 0.298872
Epoch 5
Validation binary_cross_entropy = 0.414138
Epoch 6
Validation binary_cross_entropy = 0.989933
Epoch 7
Validation binary_cross_entropy = 0.306474
Epoch 8
Validation binary_cross_entropy = 0.312818
Epoch 9
Loss = 6.8364e-01, PNorm = 63.2825, GNorm = 5.0200, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 0.934482
Epoch 10
Validation binary_cross_entropy = 0.547546
Epoch 11
Validation binary_cross_entropy = 0.309192
Epoch 12
Validation binary_cross_entropy = 0.598920
Epoch 13
Validation binary_cross_entropy = 0.757241
Epoch 14
Loss = 7.5174e-01, PNorm = 63.4329, GNorm = 4.7699, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 0.528832
Epoch 15
Validation binary_cross_entropy = 0.518389
Epoch 16
Validation binary_cross_entropy = 0.456228
Epoch 17
Validation binary_cross_entropy = 0.651107
Epoch 18
Validation binary_cross_entropy = 0.637769
Epoch 19
Loss = 7.0490e-01, PNorm = 63.5508, GNorm = 4.1631, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 0.411958
Epoch 20
Validation binary_cross_entropy = 0.379109
Epoch 21
Validation binary_cross_entropy = 0.663031
Epoch 22
Validation binary_cross_entropy = 0.654711
Epoch 23
Validation binary_cross_entropy = 0.513110
Epoch 24
Loss = 6.4934e-01, PNorm = 63.6422, GNorm = 4.8818, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 0.425396
Epoch 25
Validation binary_cross_entropy = 0.566951
Epoch 26
Validation binary_cross_entropy = 0.563345
Epoch 27
Validation binary_cross_entropy = 0.511907
Epoch 28
Validation binary_cross_entropy = 0.524909
Epoch 29
Loss = 5.9170e-01, PNorm = 63.7176, GNorm = 3.1235, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 0.557497
Epoch 30
Validation binary_cross_entropy = 0.508854
Epoch 31
Validation binary_cross_entropy = 0.505205
Epoch 32
Validation binary_cross_entropy = 0.682773
Epoch 33
Validation binary_cross_entropy = 0.551893
Epoch 34
Loss = 5.6338e-01, PNorm = 63.7647, GNorm = 2.2943, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 0.402073
Epoch 35
Validation binary_cross_entropy = 0.421800
Epoch 36
Validation binary_cross_entropy = 0.516861
Epoch 37
Validation binary_cross_entropy = 0.551832
Epoch 38
Validation binary_cross_entropy = 0.451983
Epoch 39
Loss = 7.0629e-01, PNorm = 63.8176, GNorm = 2.3840, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 0.445434
Epoch 40
Validation binary_cross_entropy = 0.500389
Epoch 41
Validation binary_cross_entropy = 0.567812
Epoch 42
Validation binary_cross_entropy = 0.546756
Epoch 43
Validation binary_cross_entropy = 0.509498
Epoch 44
Loss = 6.4506e-01, PNorm = 63.8690, GNorm = 2.4382, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 0.499064
Epoch 45
Validation binary_cross_entropy = 0.532511
Epoch 46
Validation binary_cross_entropy = 0.562847
Epoch 47
Validation binary_cross_entropy = 0.552566
Epoch 48
Validation binary_cross_entropy = 0.488790
Epoch 49
Loss = 6.0301e-01, PNorm = 63.9332, GNorm = 2.8309, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 0.509063
Epoch 50
Validation binary_cross_entropy = 0.425102
Epoch 51
Validation binary_cross_entropy = 0.462721
Epoch 52
Validation binary_cross_entropy = 0.551021
Epoch 53
Validation binary_cross_entropy = 0.589781
Epoch 54
Loss = 6.1229e-01, PNorm = 63.9844, GNorm = 2.2209, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 0.512728
Epoch 55
Validation binary_cross_entropy = 0.450623
Epoch 56
Validation binary_cross_entropy = 0.463310
Epoch 57
Validation binary_cross_entropy = 0.459335
Epoch 58
Validation binary_cross_entropy = 0.474010
Epoch 59
Loss = 5.5366e-01, PNorm = 64.0474, GNorm = 1.6926, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 0.503369
Epoch 60
Validation binary_cross_entropy = 0.505439
Epoch 61
Validation binary_cross_entropy = 0.500602
Epoch 62
Validation binary_cross_entropy = 0.497277
Epoch 63
Validation binary_cross_entropy = 0.481739
Epoch 64
Loss = 6.1077e-01, PNorm = 64.1116, GNorm = 1.5692, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 0.490339
Epoch 65
Validation binary_cross_entropy = 0.506147
Epoch 66
Validation binary_cross_entropy = 0.568908
Epoch 67
Validation binary_cross_entropy = 0.517063
Epoch 68
Validation binary_cross_entropy = 0.460587
Epoch 69
Loss = 5.2219e-01, PNorm = 64.1851, GNorm = 1.6659, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 0.461268
Epoch 70
Validation binary_cross_entropy = 0.541916
Epoch 71
Validation binary_cross_entropy = 0.515901
Epoch 72
Validation binary_cross_entropy = 0.422997
Epoch 73
Validation binary_cross_entropy = 0.424317
Epoch 74
Loss = 4.9852e-01, PNorm = 64.2593, GNorm = 2.2279, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 0.695956
Epoch 75
Validation binary_cross_entropy = 0.474141
Epoch 76
Validation binary_cross_entropy = 0.380577
Epoch 77
Validation binary_cross_entropy = 0.509671
Epoch 78
Validation binary_cross_entropy = 0.649239
Epoch 79
Loss = 5.6326e-01, PNorm = 64.3305, GNorm = 2.1418, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 0.601090
Epoch 80
Validation binary_cross_entropy = 0.461573
Epoch 81
Validation binary_cross_entropy = 0.446582
Epoch 82
Validation binary_cross_entropy = 0.552948
Epoch 83
Validation binary_cross_entropy = 0.568176
Epoch 84
Loss = 5.2070e-01, PNorm = 64.4223, GNorm = 1.5745, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 0.425082
Epoch 85
Validation binary_cross_entropy = 0.661683
Epoch 86
Validation binary_cross_entropy = 0.649528
Epoch 87
Validation binary_cross_entropy = 0.370485
Epoch 88
Validation binary_cross_entropy = 0.467967
Epoch 89
Loss = 5.2102e-01, PNorm = 64.5004, GNorm = 1.8889, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 0.570789
Epoch 90
Validation binary_cross_entropy = 0.531688
Epoch 91
Validation binary_cross_entropy = 0.448440
Epoch 92
Validation binary_cross_entropy = 0.421496
Epoch 93
Validation binary_cross_entropy = 0.372979
Epoch 94
Loss = 4.9588e-01, PNorm = 64.5905, GNorm = 1.6387, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 0.417093
Epoch 95
Validation binary_cross_entropy = 0.459382
Epoch 96
Validation binary_cross_entropy = 0.401655
Epoch 97
Validation binary_cross_entropy = 0.400023
Epoch 98
Validation binary_cross_entropy = 0.435690
Epoch 99
Loss = 6.7552e-01, PNorm = 64.6970, GNorm = 6.3244, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.360922
Epoch 100
Validation binary_cross_entropy = 0.453231
Epoch 101
Validation binary_cross_entropy = 0.692516
Epoch 102
Validation binary_cross_entropy = 0.513603
Epoch 103
Validation binary_cross_entropy = 0.398976
Epoch 104
Loss = 4.7763e-01, PNorm = 64.7862, GNorm = 1.3288, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.360552
Epoch 105
Validation binary_cross_entropy = 0.414690
Epoch 106
Validation binary_cross_entropy = 0.504618
Epoch 107
Validation binary_cross_entropy = 0.561164
Epoch 108
Validation binary_cross_entropy = 0.480846
Epoch 109
Loss = 4.2861e-01, PNorm = 64.8959, GNorm = 2.3231, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.404064
Epoch 110
Validation binary_cross_entropy = 0.435686
Epoch 111
Validation binary_cross_entropy = 0.449121
Epoch 112
Validation binary_cross_entropy = 0.405311
Epoch 113
Validation binary_cross_entropy = 0.410347
Epoch 114
Loss = 5.1305e-01, PNorm = 64.9679, GNorm = 1.6515, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.447559
Epoch 115
Validation binary_cross_entropy = 0.422570
Epoch 116
Validation binary_cross_entropy = 0.444012
Epoch 117
Validation binary_cross_entropy = 0.525262
Epoch 118
Validation binary_cross_entropy = 0.411156
Epoch 119
Loss = 4.0218e-01, PNorm = 65.0634, GNorm = 1.5317, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.405639
Epoch 120
Validation binary_cross_entropy = 0.410765
Epoch 121
Validation binary_cross_entropy = 0.427659
Epoch 122
Validation binary_cross_entropy = 0.405211
Epoch 123
Validation binary_cross_entropy = 0.442911
Epoch 124
Loss = 3.0890e-01, PNorm = 65.1671, GNorm = 1.6124, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.394777
Epoch 125
Validation binary_cross_entropy = 0.590893
Epoch 126
Validation binary_cross_entropy = 0.406036
Epoch 127
Validation binary_cross_entropy = 0.428446
Epoch 128
Validation binary_cross_entropy = 0.475835
Epoch 129
Loss = 2.7375e-01, PNorm = 65.2914, GNorm = 1.6055, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.474203
Epoch 130
Validation binary_cross_entropy = 0.435007
Epoch 131
Validation binary_cross_entropy = 0.577607
Epoch 132
Validation binary_cross_entropy = 0.355320
Epoch 133
Validation binary_cross_entropy = 0.376572
Epoch 134
Loss = 3.5452e-01, PNorm = 65.4069, GNorm = 3.2060, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.556744
Epoch 135
Validation binary_cross_entropy = 0.391409
Epoch 136
Validation binary_cross_entropy = 0.340950
Epoch 137
Validation binary_cross_entropy = 0.413061
Epoch 138
Validation binary_cross_entropy = 0.511444
Epoch 139
Loss = 2.1878e-01, PNorm = 65.5293, GNorm = 1.8538, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.610915
Epoch 140
Validation binary_cross_entropy = 0.571178
Epoch 141
Validation binary_cross_entropy = 0.543512
Epoch 142
Validation binary_cross_entropy = 0.520974
Epoch 143
Validation binary_cross_entropy = 0.574298
Epoch 144
Loss = 3.0774e-01, PNorm = 65.6556, GNorm = 2.9348, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.389428
Epoch 145
Validation binary_cross_entropy = 0.369535
Epoch 146
Validation binary_cross_entropy = 0.412468
Epoch 147
Validation binary_cross_entropy = 0.516225
Epoch 148
Validation binary_cross_entropy = 0.455079
Epoch 149
Loss = 2.4803e-01, PNorm = 65.7841, GNorm = 2.4804, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.451709
Epoch 150
Validation binary_cross_entropy = 0.481297
Epoch 151
Validation binary_cross_entropy = 0.581413
Epoch 152
Validation binary_cross_entropy = 0.496244
Epoch 153
Validation binary_cross_entropy = 0.536737
Epoch 154
Loss = 2.9935e-01, PNorm = 65.9096, GNorm = 2.0373, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.504525
Epoch 155
Validation binary_cross_entropy = 0.532952
Epoch 156
Validation binary_cross_entropy = 0.568003
Epoch 157
Validation binary_cross_entropy = 0.622010
Epoch 158
Validation binary_cross_entropy = 0.691760
Epoch 159
Loss = 2.1128e-01, PNorm = 66.0280, GNorm = 2.7696, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.680182
Epoch 160
Validation binary_cross_entropy = 0.674138
Epoch 161
Validation binary_cross_entropy = 0.728066
Epoch 162
Validation binary_cross_entropy = 0.690325
Epoch 163
Validation binary_cross_entropy = 0.691302
Epoch 164
Loss = 2.6650e-01, PNorm = 66.1355, GNorm = 3.8853, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.615440
Epoch 165
Validation binary_cross_entropy = 0.651972
Epoch 166
Validation binary_cross_entropy = 0.674294
Epoch 167
Validation binary_cross_entropy = 0.675057
Epoch 168
Validation binary_cross_entropy = 0.655449
Epoch 169
Loss = 2.4711e-01, PNorm = 66.2601, GNorm = 3.1258, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.625582
Epoch 170
Validation binary_cross_entropy = 0.685914
Epoch 171
Validation binary_cross_entropy = 0.578064
Epoch 172
Validation binary_cross_entropy = 0.594077
Epoch 173
Validation binary_cross_entropy = 0.608796
Epoch 174
Loss = 2.2821e-01, PNorm = 66.3676, GNorm = 1.6336, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 0.612264
Epoch 175
Validation binary_cross_entropy = 0.622833
Epoch 176
Validation binary_cross_entropy = 0.655566
Epoch 177
Validation binary_cross_entropy = 0.730178
Epoch 178
Validation binary_cross_entropy = 0.675535
Epoch 179
Loss = 2.1640e-01, PNorm = 66.4821, GNorm = 3.0202, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 0.635750
Epoch 180
Validation binary_cross_entropy = 0.631667
Epoch 181
Validation binary_cross_entropy = 0.655540
Epoch 182
Validation binary_cross_entropy = 0.664697
Epoch 183
Validation binary_cross_entropy = 0.668432
Epoch 184
Loss = 1.1927e-01, PNorm = 66.5751, GNorm = 1.7089, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 0.679258
Epoch 185
Validation binary_cross_entropy = 0.718565
Epoch 186
Validation binary_cross_entropy = 0.710256
Epoch 187
Validation binary_cross_entropy = 0.765935
Epoch 188
Validation binary_cross_entropy = 0.761811
Epoch 189
Loss = 1.4232e-01, PNorm = 66.6724, GNorm = 1.1739, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 0.659185
Epoch 190
Validation binary_cross_entropy = 0.652364
Epoch 191
Validation binary_cross_entropy = 0.806517
Epoch 192
Validation binary_cross_entropy = 0.730548
Epoch 193
Validation binary_cross_entropy = 0.758978
Epoch 194
Loss = 1.6751e-01, PNorm = 66.7763, GNorm = 3.7780, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 0.785842
Epoch 195
Validation binary_cross_entropy = 0.761353
Epoch 196
Validation binary_cross_entropy = 0.827404
Epoch 197
Validation binary_cross_entropy = 0.715253
Epoch 198
Validation binary_cross_entropy = 0.685623
Epoch 199
Loss = 1.3429e-01, PNorm = 66.8715, GNorm = 2.4346, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 0.727729
Epoch 200
Validation binary_cross_entropy = 0.787337
Epoch 201
Validation binary_cross_entropy = 0.888449
Epoch 202
Validation binary_cross_entropy = 0.862079
Epoch 203
Validation binary_cross_entropy = 0.858547
Epoch 204
Loss = 2.3100e-01, PNorm = 66.9509, GNorm = 1.2725, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 0.952700
Epoch 205
Validation binary_cross_entropy = 0.940463
Epoch 206
Validation binary_cross_entropy = 0.833851
Epoch 207
Validation binary_cross_entropy = 0.856566
Epoch 208
Validation binary_cross_entropy = 0.805459
Epoch 209
Loss = 4.8214e-02, PNorm = 67.0231, GNorm = 0.6904, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 0.947593
Epoch 210
Validation binary_cross_entropy = 0.856513
Epoch 211
Validation binary_cross_entropy = 0.780611
Epoch 212
Validation binary_cross_entropy = 0.789957
Epoch 213
Validation binary_cross_entropy = 0.844410
Epoch 214
Loss = 9.6699e-02, PNorm = 67.0992, GNorm = 1.8094, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 0.959258
Epoch 215
Validation binary_cross_entropy = 0.997409
Epoch 216
Validation binary_cross_entropy = 1.025235
Epoch 217
Validation binary_cross_entropy = 1.052710
Epoch 218
Validation binary_cross_entropy = 1.092445
Epoch 219
Loss = 7.7272e-02, PNorm = 67.1685, GNorm = 0.8569, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 1.139068
Epoch 220
Validation binary_cross_entropy = 1.094252
Epoch 221
Validation binary_cross_entropy = 1.032983
Epoch 222
Validation binary_cross_entropy = 1.005158
Epoch 223
Validation binary_cross_entropy = 0.996140
Epoch 224
Loss = 1.1100e-01, PNorm = 67.2390, GNorm = 0.7395, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 1.004528
Epoch 225
Validation binary_cross_entropy = 1.010194
Epoch 226
Validation binary_cross_entropy = 1.001788
Epoch 227
Validation binary_cross_entropy = 1.081595
Epoch 228
Validation binary_cross_entropy = 1.133931
Epoch 229
Loss = 1.4656e-01, PNorm = 67.3158, GNorm = 0.9273, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 0.923799
Epoch 230
Validation binary_cross_entropy = 0.903259
Epoch 231
Validation binary_cross_entropy = 0.943619
Epoch 232
Validation binary_cross_entropy = 0.983836
Epoch 233
Validation binary_cross_entropy = 0.760726
Epoch 234
Loss = 1.0202e-01, PNorm = 67.3863, GNorm = 2.3109, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 0.782940
Epoch 235
Validation binary_cross_entropy = 0.754580
Epoch 236
Validation binary_cross_entropy = 0.827625
Epoch 237
Validation binary_cross_entropy = 0.960367
Epoch 238
Validation binary_cross_entropy = 0.967375
Epoch 239
Loss = 9.3937e-02, PNorm = 67.4737, GNorm = 1.4101, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 0.976392
Epoch 240
Validation binary_cross_entropy = 1.004143
Epoch 241
Validation binary_cross_entropy = 1.125548
Epoch 242
Validation binary_cross_entropy = 1.304341
Epoch 243
Validation binary_cross_entropy = 1.015541
Epoch 244
Loss = 1.0096e-01, PNorm = 67.5400, GNorm = 0.7349, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 0.929505
Epoch 245
Validation binary_cross_entropy = 0.912798
Epoch 246
Validation binary_cross_entropy = 0.920925
Epoch 247
Validation binary_cross_entropy = 1.021226
Epoch 248
Validation binary_cross_entropy = 0.999889
Epoch 249
Loss = 5.1999e-02, PNorm = 67.6054, GNorm = 1.0720, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 0.903183
Epoch 250
Validation binary_cross_entropy = 0.923696
Epoch 251
Validation binary_cross_entropy = 1.000984
Epoch 252
Validation binary_cross_entropy = 0.945743
Epoch 253
Validation binary_cross_entropy = 0.879110
Epoch 254
Loss = 7.3290e-02, PNorm = 67.6626, GNorm = 0.6395, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 0.863368
Epoch 255
Validation binary_cross_entropy = 0.880684
Epoch 256
Validation binary_cross_entropy = 0.906490
Epoch 257
Validation binary_cross_entropy = 0.933370
Epoch 258
Validation binary_cross_entropy = 0.962841
Epoch 259
Loss = 5.8618e-02, PNorm = 67.7211, GNorm = 0.3700, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 1.029311
Epoch 260
Validation binary_cross_entropy = 1.102662
Epoch 261
Validation binary_cross_entropy = 1.198411
Epoch 262
Validation binary_cross_entropy = 1.314830
Epoch 263
Validation binary_cross_entropy = 1.335524
Epoch 264
Loss = 3.7244e-02, PNorm = 67.7764, GNorm = 0.2778, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 1.234284
Epoch 265
Validation binary_cross_entropy = 1.247617
Epoch 266
Validation binary_cross_entropy = 1.204413
Epoch 267
Validation binary_cross_entropy = 1.306379
Epoch 268
Validation binary_cross_entropy = 1.207617
Epoch 269
Loss = 5.7970e-02, PNorm = 67.8184, GNorm = 1.6239, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 1.115563
Epoch 270
Validation binary_cross_entropy = 1.126639
Epoch 271
Validation binary_cross_entropy = 1.175565
Epoch 272
Validation binary_cross_entropy = 1.278191
Epoch 273
Validation binary_cross_entropy = 1.281467
Epoch 274
Loss = 6.7799e-02, PNorm = 67.8727, GNorm = 1.2204, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 1.275515
Epoch 275
Validation binary_cross_entropy = 1.318951
Epoch 276
Validation binary_cross_entropy = 1.365902
Epoch 277
Validation binary_cross_entropy = 1.440852
Epoch 278
Validation binary_cross_entropy = 1.441641
Epoch 279
Loss = 5.7664e-02, PNorm = 67.9232, GNorm = 1.5768, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 1.499216
Epoch 280
Validation binary_cross_entropy = 1.490388
Epoch 281
Validation binary_cross_entropy = 1.520262
Epoch 282
Validation binary_cross_entropy = 1.535105
Epoch 283
Validation binary_cross_entropy = 1.727079
Epoch 284
Loss = 2.7786e-02, PNorm = 67.9711, GNorm = 2.1677, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 1.763739
Epoch 285
Validation binary_cross_entropy = 1.645660
Epoch 286
Validation binary_cross_entropy = 1.650072
Epoch 287
Validation binary_cross_entropy = 1.713339
Epoch 288
Validation binary_cross_entropy = 1.795808
Epoch 289
Loss = 1.5754e-02, PNorm = 68.0131, GNorm = 2.4261, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 1.862291
Epoch 290
Validation binary_cross_entropy = 1.805245
Epoch 291
Validation binary_cross_entropy = 1.762914
Epoch 292
Validation binary_cross_entropy = 1.975118
Epoch 293
Validation binary_cross_entropy = 2.003293
Epoch 294
Loss = 7.3219e-02, PNorm = 68.0479, GNorm = 3.3368, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 1.808892
Epoch 295
Validation binary_cross_entropy = 1.630189
Epoch 296
Validation binary_cross_entropy = 1.545518
Epoch 297
Validation binary_cross_entropy = 1.543763
Epoch 298
Validation binary_cross_entropy = 1.596632
Epoch 299
Loss = 9.9779e-03, PNorm = 68.0834, GNorm = 0.4457, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 1.636361
Model 0 best validation binary_cross_entropy = 0.288900 on epoch 0
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.222776
Ensemble test binary_cross_entropy = 0.222776
10-fold cross validation
	Seed 0 ==> test binary_cross_entropy = 0.242493
	Seed 1 ==> test binary_cross_entropy = 0.198337
	Seed 2 ==> test binary_cross_entropy = 0.205699
	Seed 3 ==> test binary_cross_entropy = 0.244754
	Seed 4 ==> test binary_cross_entropy = 0.193636
	Seed 5 ==> test binary_cross_entropy = 0.211122
	Seed 6 ==> test binary_cross_entropy = 0.227363
	Seed 7 ==> test binary_cross_entropy = 0.223729
	Seed 8 ==> test binary_cross_entropy = 0.225327
	Seed 9 ==> test binary_cross_entropy = 0.222776
Overall test binary_cross_entropy = 0.219524 +/- 0.016335
Elapsed time = 0:06:37
